Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X........X...........................................] duration: 5.905534529s FAIL (2025-08-07 19:38:10 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.905556833s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X........X..........................................] duration: 5.634785656s FAIL (2025-08-07 19:37:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.634820374s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................................X........X......] duration: 1.902659449s FAIL (2025-08-07 19:37:23 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== m_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Aug 7 19:28:33.238514538 (9h48m32.827859525s) [upload=10221]: Starting upload Aug 7 19:28:33.238562180 (9h48m32.827907157s) [upload=10221]: Done starting upload Aug 7 19:28:33.238627795 (9h48m32.827972782s) [upload=10221]: Uploading segments... Aug 7 19:28:33.238663914 (9h48m32.828008891s) [upload=10221]: Got next segment. Inline: true Aug 7 19:28:33.238667501 (9h48m32.828012488s) [upload=10221]: Waiting for error group managing segments... Aug 7 19:28:33.238704382 (9h48m32.828049429s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:33.238711796 (9h48m32.828056783s) [upload=10221]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:33.238715493 (9h48m32.82806049s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:33.658266417 (9h48m33.247611404s) [upload=10221]: Done uploading segments... Aug 7 19:28:33.658283299 (9h48m33.247628286s) [upload=10221]: Upload finished. err: <nil> Aug 7 19:28:51.361704008 (9h48m50.951048995s) [upload=10225]: Starting upload Aug 7 19:28:51.361752521 (9h48m50.951097508s) [upload=10225]: Done starting upload Aug 7 19:28:51.361814239 (9h48m50.951159226s) [upload=10225]: Uploading segments... Aug 7 19:28:51.361846150 (9h48m50.951191137s) [upload=10225]: Got next segment. Inline: true Aug 7 19:28:51.361849707 (9h48m50.951194695s) [upload=10225]: Waiting for error group managing segments... Aug 7 19:28:51.361880727 (9h48m50.951225714s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:51.361884544 (9h48m50.951229531s) [upload=10225]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:51.361886908 (9h48m50.951231895s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:51.634702903 (9h48m51.22404789s) [upload=10225]: Done uploading segments... Aug 7 19:28:51.634737349 (9h48m51.224082326s) [upload=10225]: Upload finished. err: <nil> Aug 7 19:29:12.220554896 (9h49m11.809899883s) [upload=10233]: Starting upload Aug 7 19:29:12.220627544 (9h49m11.809972551s) [upload=10233]: Done starting upload Aug 7 19:29:12.220708579 (9h49m11.810053576s) [upload=10233]: Uploading segments... Aug 7 19:29:12.220803531 (9h49m11.810148518s) [upload=10233]: Got next segment. Inline: true Aug 7 19:29:12.220809533 (9h49m11.81015452s) [upload=10233]: Waiting for error group managing segments... Aug 7 19:29:12.220871651 (9h49m11.810216638s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:12.220877713 (9h49m11.810222701s) [upload=10233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:12.220880960 (9h49m11.810225947s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:12.573045808 (9h49m12.162390795s) [upload=10233]: Done uploading segments... Aug 7 19:29:12.573091345 (9h49m12.162436332s) [upload=10233]: Upload finished. err: <nil> Aug 7 19:29:41.735121145 (9h49m41.324466132s) [upload=10244]: Starting upload Aug 7 19:29:41.735168356 (9h49m41.324513363s) [upload=10244]: Done starting upload Aug 7 19:29:41.735255312 (9h49m41.324600289s) [upload=10244]: Uploading segments... Aug 7 19:29:41.735311500 (9h49m41.324656487s) [upload=10244]: Got next segment. Inline: true Aug 7 19:29:41.735314926 (9h49m41.324659913s) [upload=10244]: Waiting for error group managing segments... Aug 7 19:29:41.735368448 (9h49m41.324713435s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:41.735372626 (9h49m41.324717613s) [upload=10244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:41.735374870 (9h49m41.324719858s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:42.000903022 (9h49m41.590248009s) [upload=10244]: Done uploading segments... Aug 7 19:29:42.000921688 (9h49m41.590266675s) [upload=10244]: Upload finished. err: <nil> Aug 7 19:30:09.357577819 (9h50m8.946922806s) [upload=10251]: Starting upload Aug 7 19:30:09.357623787 (9h50m8.946968774s) [upload=10251]: Done starting upload Aug 7 19:30:09.357706075 (9h50m8.947051062s) [upload=10251]: Uploading segments... Aug 7 19:30:09.357828999 (9h50m8.947173986s) [upload=10251]: Got next segment. Inline: true Aug 7 19:30:09.357834209 (9h50m8.947179196s) [upload=10251]: Waiting for error group managing segments... Aug 7 19:30:09.357879085 (9h50m8.947224072s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:09.357883534 (9h50m8.947228521s) [upload=10251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:09.357885748 (9h50m8.947230735s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:09.831093883 (9h50m9.42043887s) [upload=10251]: Done uploading segments... Aug 7 19:30:09.831111727 (9h50m9.420456714s) [upload=10251]: Upload finished. err: <nil> Aug 7 19:30:31.706676511 (9h50m31.296021498s) [upload=10259]: Starting upload Aug 7 19:30:31.706715425 (9h50m31.296060413s) [upload=10259]: Done starting upload Aug 7 19:30:31.706962198 (9h50m31.296307185s) [upload=10259]: Uploading segments... Aug 7 19:30:31.706996403 (9h50m31.29634138s) [upload=10259]: Got next segment. Inline: true Aug 7 19:30:31.707006341 (9h50m31.296351329s) [upload=10259]: Waiting for error group managing segments... Aug 7 19:30:31.707042211 (9h50m31.296387198s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:31.707046168 (9h50m31.296391155s) [upload=10259]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:31.707049004 (9h50m31.296393991s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:32.134768857 (9h50m31.724113844s) [upload=10259]: Done uploading segments... Aug 7 19:30:32.134809155 (9h50m31.724154142s) [upload=10259]: Upload finished. err: <nil> Aug 7 19:31:14.642237281 (9h51m14.231582268s) [upload=10273]: Starting upload Aug 7 19:31:14.642283008 (9h51m14.231627995s) [upload=10273]: Done starting upload Aug 7 19:31:14.642334347 (9h51m14.231679324s) [upload=10273]: Uploading segments... Aug 7 19:31:14.642388370 (9h51m14.231733357s) [upload=10273]: Got next segment. Inline: true Aug 7 19:31:14.642395824 (9h51m14.231740841s) [upload=10273]: Waiting for error group managing segments... Aug 7 19:31:14.642450859 (9h51m14.231795876s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:14.642458204 (9h51m14.231803231s) [upload=10273]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:14.642463464 (9h51m14.231808481s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:14.916538458 (9h51m14.505883465s) [upload=10273]: Done uploading segments... Aug 7 19:31:14.916565079 (9h51m14.505910066s) [upload=10273]: Upload finished. err: <nil> Aug 7 19:31:59.580470078 (9h51m59.169815065s) [upload=10283]: Starting upload Aug 7 19:31:59.580508693 (9h51m59.16985368s) [upload=10283]: Done starting upload Aug 7 19:31:59.580563077 (9h51m59.169908064s) [upload=10283]: Uploading segments... Aug 7 19:31:59.580599707 (9h51m59.169944694s) [upload=10283]: Got next segment. Inline: true Aug 7 19:31:59.580603684 (9h51m59.169948671s) [upload=10283]: Waiting for error group managing segments... Aug 7 19:31:59.580637348 (9h51m59.169982335s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:59.580640916 (9h51m59.169985893s) [upload=10283]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:59.580643240 (9h51m59.169988227s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:59.894471131 (9h51m59.483816118s) [upload=10283]: Done uploading segments... Aug 7 19:31:59.894488253 (9h51m59.48383323s) [upload=10283]: Upload finished. err: <nil> Aug 7 19:32:38.308640915 (9h52m37.897985902s) [upload=10295]: Starting upload Aug 7 19:32:38.308709886 (9h52m37.898054873s) [upload=10295]: Done starting upload Aug 7 19:32:38.308805570 (9h52m37.898150557s) [upload=10295]: Uploading segments... Aug 7 19:32:38.308879861 (9h52m37.898224838s) [upload=10295]: Got next segment. Inline: true Aug 7 19:32:38.308884580 (9h52m37.898229567s) [upload=10295]: Waiting for error group managing segments... Aug 7 19:32:38.308924838 (9h52m37.898269825s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:32:38.308929156 (9h52m37.898274143s) [upload=10295]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:32:38.308931530 (9h52m37.898276517s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:32:38.590415960 (9h52m38.179760957s) [upload=10295]: Done uploading segments... Aug 7 19:32:38.590438433 (9h52m38.179783421s) [upload=10295]: Upload finished. err: <nil> Duration: 1.902684758s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X........X.....] duration: 2.115416261s FAIL (2025-08-07 19:37:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== m_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Aug 7 19:28:33.238514538 (9h48m32.827859525s) [upload=10221]: Starting upload Aug 7 19:28:33.238562180 (9h48m32.827907157s) [upload=10221]: Done starting upload Aug 7 19:28:33.238627795 (9h48m32.827972782s) [upload=10221]: Uploading segments... Aug 7 19:28:33.238663914 (9h48m32.828008891s) [upload=10221]: Got next segment. Inline: true Aug 7 19:28:33.238667501 (9h48m32.828012488s) [upload=10221]: Waiting for error group managing segments... Aug 7 19:28:33.238704382 (9h48m32.828049429s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:33.238711796 (9h48m32.828056783s) [upload=10221]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:33.238715493 (9h48m32.82806049s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:33.658266417 (9h48m33.247611404s) [upload=10221]: Done uploading segments... Aug 7 19:28:33.658283299 (9h48m33.247628286s) [upload=10221]: Upload finished. err: <nil> Aug 7 19:28:51.361704008 (9h48m50.951048995s) [upload=10225]: Starting upload Aug 7 19:28:51.361752521 (9h48m50.951097508s) [upload=10225]: Done starting upload Aug 7 19:28:51.361814239 (9h48m50.951159226s) [upload=10225]: Uploading segments... Aug 7 19:28:51.361846150 (9h48m50.951191137s) [upload=10225]: Got next segment. Inline: true Aug 7 19:28:51.361849707 (9h48m50.951194695s) [upload=10225]: Waiting for error group managing segments... Aug 7 19:28:51.361880727 (9h48m50.951225714s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:51.361884544 (9h48m50.951229531s) [upload=10225]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:51.361886908 (9h48m50.951231895s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:51.634702903 (9h48m51.22404789s) [upload=10225]: Done uploading segments... Aug 7 19:28:51.634737349 (9h48m51.224082326s) [upload=10225]: Upload finished. err: <nil> Aug 7 19:29:12.220554896 (9h49m11.809899883s) [upload=10233]: Starting upload Aug 7 19:29:12.220627544 (9h49m11.809972551s) [upload=10233]: Done starting upload Aug 7 19:29:12.220708579 (9h49m11.810053576s) [upload=10233]: Uploading segments... Aug 7 19:29:12.220803531 (9h49m11.810148518s) [upload=10233]: Got next segment. Inline: true Aug 7 19:29:12.220809533 (9h49m11.81015452s) [upload=10233]: Waiting for error group managing segments... Aug 7 19:29:12.220871651 (9h49m11.810216638s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:12.220877713 (9h49m11.810222701s) [upload=10233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:12.220880960 (9h49m11.810225947s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:12.573045808 (9h49m12.162390795s) [upload=10233]: Done uploading segments... Aug 7 19:29:12.573091345 (9h49m12.162436332s) [upload=10233]: Upload finished. err: <nil> Aug 7 19:29:41.735121145 (9h49m41.324466132s) [upload=10244]: Starting upload Aug 7 19:29:41.735168356 (9h49m41.324513363s) [upload=10244]: Done starting upload Aug 7 19:29:41.735255312 (9h49m41.324600289s) [upload=10244]: Uploading segments... Aug 7 19:29:41.735311500 (9h49m41.324656487s) [upload=10244]: Got next segment. Inline: true Aug 7 19:29:41.735314926 (9h49m41.324659913s) [upload=10244]: Waiting for error group managing segments... Aug 7 19:29:41.735368448 (9h49m41.324713435s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:41.735372626 (9h49m41.324717613s) [upload=10244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:41.735374870 (9h49m41.324719858s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:42.000903022 (9h49m41.590248009s) [upload=10244]: Done uploading segments... Aug 7 19:29:42.000921688 (9h49m41.590266675s) [upload=10244]: Upload finished. err: <nil> Aug 7 19:30:09.357577819 (9h50m8.946922806s) [upload=10251]: Starting upload Aug 7 19:30:09.357623787 (9h50m8.946968774s) [upload=10251]: Done starting upload Aug 7 19:30:09.357706075 (9h50m8.947051062s) [upload=10251]: Uploading segments... Aug 7 19:30:09.357828999 (9h50m8.947173986s) [upload=10251]: Got next segment. Inline: true Aug 7 19:30:09.357834209 (9h50m8.947179196s) [upload=10251]: Waiting for error group managing segments... Aug 7 19:30:09.357879085 (9h50m8.947224072s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:09.357883534 (9h50m8.947228521s) [upload=10251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:09.357885748 (9h50m8.947230735s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:09.831093883 (9h50m9.42043887s) [upload=10251]: Done uploading segments... Aug 7 19:30:09.831111727 (9h50m9.420456714s) [upload=10251]: Upload finished. err: <nil> Aug 7 19:30:31.706676511 (9h50m31.296021498s) [upload=10259]: Starting upload Aug 7 19:30:31.706715425 (9h50m31.296060413s) [upload=10259]: Done starting upload Aug 7 19:30:31.706962198 (9h50m31.296307185s) [upload=10259]: Uploading segments... Aug 7 19:30:31.706996403 (9h50m31.29634138s) [upload=10259]: Got next segment. Inline: true Aug 7 19:30:31.707006341 (9h50m31.296351329s) [upload=10259]: Waiting for error group managing segments... Aug 7 19:30:31.707042211 (9h50m31.296387198s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:31.707046168 (9h50m31.296391155s) [upload=10259]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:31.707049004 (9h50m31.296393991s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:32.134768857 (9h50m31.724113844s) [upload=10259]: Done uploading segments... Aug 7 19:30:32.134809155 (9h50m31.724154142s) [upload=10259]: Upload finished. err: <nil> Aug 7 19:31:14.642237281 (9h51m14.231582268s) [upload=10273]: Starting upload Aug 7 19:31:14.642283008 (9h51m14.231627995s) [upload=10273]: Done starting upload Aug 7 19:31:14.642334347 (9h51m14.231679324s) [upload=10273]: Uploading segments... Aug 7 19:31:14.642388370 (9h51m14.231733357s) [upload=10273]: Got next segment. Inline: true Aug 7 19:31:14.642395824 (9h51m14.231740841s) [upload=10273]: Waiting for error group managing segments... Aug 7 19:31:14.642450859 (9h51m14.231795876s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:14.642458204 (9h51m14.231803231s) [upload=10273]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:14.642463464 (9h51m14.231808481s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:14.916538458 (9h51m14.505883465s) [upload=10273]: Done uploading segments... Aug 7 19:31:14.916565079 (9h51m14.505910066s) [upload=10273]: Upload finished. err: <nil> Aug 7 19:31:59.580470078 (9h51m59.169815065s) [upload=10283]: Starting upload Aug 7 19:31:59.580508693 (9h51m59.16985368s) [upload=10283]: Done starting upload Aug 7 19:31:59.580563077 (9h51m59.169908064s) [upload=10283]: Uploading segments... Aug 7 19:31:59.580599707 (9h51m59.169944694s) [upload=10283]: Got next segment. Inline: true Aug 7 19:31:59.580603684 (9h51m59.169948671s) [upload=10283]: Waiting for error group managing segments... Aug 7 19:31:59.580637348 (9h51m59.169982335s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:59.580640916 (9h51m59.169985893s) [upload=10283]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:59.580643240 (9h51m59.169988227s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:59.894471131 (9h51m59.483816118s) [upload=10283]: Done uploading segments... Aug 7 19:31:59.894488253 (9h51m59.48383323s) [upload=10283]: Upload finished. err: <nil> Aug 7 19:32:38.308640915 (9h52m37.897985902s) [upload=10295]: Starting upload Aug 7 19:32:38.308709886 (9h52m37.898054873s) [upload=10295]: Done starting upload Aug 7 19:32:38.308805570 (9h52m37.898150557s) [upload=10295]: Uploading segments... Aug 7 19:32:38.308879861 (9h52m37.898224838s) [upload=10295]: Got next segment. Inline: true Aug 7 19:32:38.308884580 (9h52m37.898229567s) [upload=10295]: Waiting for error group managing segments... Aug 7 19:32:38.308924838 (9h52m37.898269825s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:32:38.308929156 (9h52m37.898274143s) [upload=10295]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:32:38.308931530 (9h52m37.898276517s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:32:38.590415960 (9h52m38.179760957s) [upload=10295]: Done uploading segments... Aug 7 19:32:38.590438433 (9h52m38.179783421s) [upload=10295]: Upload finished. err: <nil> Duration: 2.11544696s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X........X.........................................] duration: 6.162632858s FAIL (2025-08-07 19:36:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 6.162657747s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X........X....] duration: 1.966527139s FAIL (2025-08-07 19:36:13 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== m_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Aug 7 19:28:33.238514538 (9h48m32.827859525s) [upload=10221]: Starting upload Aug 7 19:28:33.238562180 (9h48m32.827907157s) [upload=10221]: Done starting upload Aug 7 19:28:33.238627795 (9h48m32.827972782s) [upload=10221]: Uploading segments... Aug 7 19:28:33.238663914 (9h48m32.828008891s) [upload=10221]: Got next segment. Inline: true Aug 7 19:28:33.238667501 (9h48m32.828012488s) [upload=10221]: Waiting for error group managing segments... Aug 7 19:28:33.238704382 (9h48m32.828049429s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:33.238711796 (9h48m32.828056783s) [upload=10221]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:33.238715493 (9h48m32.82806049s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:33.658266417 (9h48m33.247611404s) [upload=10221]: Done uploading segments... Aug 7 19:28:33.658283299 (9h48m33.247628286s) [upload=10221]: Upload finished. err: <nil> Aug 7 19:28:51.361704008 (9h48m50.951048995s) [upload=10225]: Starting upload Aug 7 19:28:51.361752521 (9h48m50.951097508s) [upload=10225]: Done starting upload Aug 7 19:28:51.361814239 (9h48m50.951159226s) [upload=10225]: Uploading segments... Aug 7 19:28:51.361846150 (9h48m50.951191137s) [upload=10225]: Got next segment. Inline: true Aug 7 19:28:51.361849707 (9h48m50.951194695s) [upload=10225]: Waiting for error group managing segments... Aug 7 19:28:51.361880727 (9h48m50.951225714s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:51.361884544 (9h48m50.951229531s) [upload=10225]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:51.361886908 (9h48m50.951231895s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:51.634702903 (9h48m51.22404789s) [upload=10225]: Done uploading segments... Aug 7 19:28:51.634737349 (9h48m51.224082326s) [upload=10225]: Upload finished. err: <nil> Aug 7 19:29:12.220554896 (9h49m11.809899883s) [upload=10233]: Starting upload Aug 7 19:29:12.220627544 (9h49m11.809972551s) [upload=10233]: Done starting upload Aug 7 19:29:12.220708579 (9h49m11.810053576s) [upload=10233]: Uploading segments... Aug 7 19:29:12.220803531 (9h49m11.810148518s) [upload=10233]: Got next segment. Inline: true Aug 7 19:29:12.220809533 (9h49m11.81015452s) [upload=10233]: Waiting for error group managing segments... Aug 7 19:29:12.220871651 (9h49m11.810216638s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:12.220877713 (9h49m11.810222701s) [upload=10233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:12.220880960 (9h49m11.810225947s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:12.573045808 (9h49m12.162390795s) [upload=10233]: Done uploading segments... Aug 7 19:29:12.573091345 (9h49m12.162436332s) [upload=10233]: Upload finished. err: <nil> Aug 7 19:29:41.735121145 (9h49m41.324466132s) [upload=10244]: Starting upload Aug 7 19:29:41.735168356 (9h49m41.324513363s) [upload=10244]: Done starting upload Aug 7 19:29:41.735255312 (9h49m41.324600289s) [upload=10244]: Uploading segments... Aug 7 19:29:41.735311500 (9h49m41.324656487s) [upload=10244]: Got next segment. Inline: true Aug 7 19:29:41.735314926 (9h49m41.324659913s) [upload=10244]: Waiting for error group managing segments... Aug 7 19:29:41.735368448 (9h49m41.324713435s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:41.735372626 (9h49m41.324717613s) [upload=10244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:41.735374870 (9h49m41.324719858s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:42.000903022 (9h49m41.590248009s) [upload=10244]: Done uploading segments... Aug 7 19:29:42.000921688 (9h49m41.590266675s) [upload=10244]: Upload finished. err: <nil> Aug 7 19:30:09.357577819 (9h50m8.946922806s) [upload=10251]: Starting upload Aug 7 19:30:09.357623787 (9h50m8.946968774s) [upload=10251]: Done starting upload Aug 7 19:30:09.357706075 (9h50m8.947051062s) [upload=10251]: Uploading segments... Aug 7 19:30:09.357828999 (9h50m8.947173986s) [upload=10251]: Got next segment. Inline: true Aug 7 19:30:09.357834209 (9h50m8.947179196s) [upload=10251]: Waiting for error group managing segments... Aug 7 19:30:09.357879085 (9h50m8.947224072s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:09.357883534 (9h50m8.947228521s) [upload=10251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:09.357885748 (9h50m8.947230735s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:09.831093883 (9h50m9.42043887s) [upload=10251]: Done uploading segments... Aug 7 19:30:09.831111727 (9h50m9.420456714s) [upload=10251]: Upload finished. err: <nil> Aug 7 19:30:31.706676511 (9h50m31.296021498s) [upload=10259]: Starting upload Aug 7 19:30:31.706715425 (9h50m31.296060413s) [upload=10259]: Done starting upload Aug 7 19:30:31.706962198 (9h50m31.296307185s) [upload=10259]: Uploading segments... Aug 7 19:30:31.706996403 (9h50m31.29634138s) [upload=10259]: Got next segment. Inline: true Aug 7 19:30:31.707006341 (9h50m31.296351329s) [upload=10259]: Waiting for error group managing segments... Aug 7 19:30:31.707042211 (9h50m31.296387198s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:31.707046168 (9h50m31.296391155s) [upload=10259]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:31.707049004 (9h50m31.296393991s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:32.134768857 (9h50m31.724113844s) [upload=10259]: Done uploading segments... Aug 7 19:30:32.134809155 (9h50m31.724154142s) [upload=10259]: Upload finished. err: <nil> Aug 7 19:31:14.642237281 (9h51m14.231582268s) [upload=10273]: Starting upload Aug 7 19:31:14.642283008 (9h51m14.231627995s) [upload=10273]: Done starting upload Aug 7 19:31:14.642334347 (9h51m14.231679324s) [upload=10273]: Uploading segments... Aug 7 19:31:14.642388370 (9h51m14.231733357s) [upload=10273]: Got next segment. Inline: true Aug 7 19:31:14.642395824 (9h51m14.231740841s) [upload=10273]: Waiting for error group managing segments... Aug 7 19:31:14.642450859 (9h51m14.231795876s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:14.642458204 (9h51m14.231803231s) [upload=10273]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:14.642463464 (9h51m14.231808481s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:14.916538458 (9h51m14.505883465s) [upload=10273]: Done uploading segments... Aug 7 19:31:14.916565079 (9h51m14.505910066s) [upload=10273]: Upload finished. err: <nil> Aug 7 19:31:59.580470078 (9h51m59.169815065s) [upload=10283]: Starting upload Aug 7 19:31:59.580508693 (9h51m59.16985368s) [upload=10283]: Done starting upload Aug 7 19:31:59.580563077 (9h51m59.169908064s) [upload=10283]: Uploading segments... Aug 7 19:31:59.580599707 (9h51m59.169944694s) [upload=10283]: Got next segment. Inline: true Aug 7 19:31:59.580603684 (9h51m59.169948671s) [upload=10283]: Waiting for error group managing segments... Aug 7 19:31:59.580637348 (9h51m59.169982335s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:59.580640916 (9h51m59.169985893s) [upload=10283]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:59.580643240 (9h51m59.169988227s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:59.894471131 (9h51m59.483816118s) [upload=10283]: Done uploading segments... Aug 7 19:31:59.894488253 (9h51m59.48383323s) [upload=10283]: Upload finished. err: <nil> Aug 7 19:32:38.308640915 (9h52m37.897985902s) [upload=10295]: Starting upload Aug 7 19:32:38.308709886 (9h52m37.898054873s) [upload=10295]: Done starting upload Aug 7 19:32:38.308805570 (9h52m37.898150557s) [upload=10295]: Uploading segments... Aug 7 19:32:38.308879861 (9h52m37.898224838s) [upload=10295]: Got next segment. Inline: true Aug 7 19:32:38.308884580 (9h52m37.898229567s) [upload=10295]: Waiting for error group managing segments... Aug 7 19:32:38.308924838 (9h52m37.898269825s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:32:38.308929156 (9h52m37.898274143s) [upload=10295]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:32:38.308931530 (9h52m37.898276517s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:32:38.590415960 (9h52m38.179760957s) [upload=10295]: Done uploading segments... Aug 7 19:32:38.590438433 (9h52m38.179783421s) [upload=10295]: Upload finished. err: <nil> Duration: 1.966553519s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X........X........................................] duration: 6.064088179s FAIL (2025-08-07 19:35:32 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 6.064113656s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X........X.......................................] duration: 5.957588102s FAIL (2025-08-07 19:35:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.957614733s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X........X...] duration: 1.849666718s FAIL (2025-08-07 19:35:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== m_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Aug 7 19:28:33.238514538 (9h48m32.827859525s) [upload=10221]: Starting upload Aug 7 19:28:33.238562180 (9h48m32.827907157s) [upload=10221]: Done starting upload Aug 7 19:28:33.238627795 (9h48m32.827972782s) [upload=10221]: Uploading segments... Aug 7 19:28:33.238663914 (9h48m32.828008891s) [upload=10221]: Got next segment. Inline: true Aug 7 19:28:33.238667501 (9h48m32.828012488s) [upload=10221]: Waiting for error group managing segments... Aug 7 19:28:33.238704382 (9h48m32.828049429s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:33.238711796 (9h48m32.828056783s) [upload=10221]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:33.238715493 (9h48m32.82806049s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:33.658266417 (9h48m33.247611404s) [upload=10221]: Done uploading segments... Aug 7 19:28:33.658283299 (9h48m33.247628286s) [upload=10221]: Upload finished. err: <nil> Aug 7 19:28:51.361704008 (9h48m50.951048995s) [upload=10225]: Starting upload Aug 7 19:28:51.361752521 (9h48m50.951097508s) [upload=10225]: Done starting upload Aug 7 19:28:51.361814239 (9h48m50.951159226s) [upload=10225]: Uploading segments... Aug 7 19:28:51.361846150 (9h48m50.951191137s) [upload=10225]: Got next segment. Inline: true Aug 7 19:28:51.361849707 (9h48m50.951194695s) [upload=10225]: Waiting for error group managing segments... Aug 7 19:28:51.361880727 (9h48m50.951225714s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:51.361884544 (9h48m50.951229531s) [upload=10225]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:51.361886908 (9h48m50.951231895s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:51.634702903 (9h48m51.22404789s) [upload=10225]: Done uploading segments... Aug 7 19:28:51.634737349 (9h48m51.224082326s) [upload=10225]: Upload finished. err: <nil> Aug 7 19:29:12.220554896 (9h49m11.809899883s) [upload=10233]: Starting upload Aug 7 19:29:12.220627544 (9h49m11.809972551s) [upload=10233]: Done starting upload Aug 7 19:29:12.220708579 (9h49m11.810053576s) [upload=10233]: Uploading segments... Aug 7 19:29:12.220803531 (9h49m11.810148518s) [upload=10233]: Got next segment. Inline: true Aug 7 19:29:12.220809533 (9h49m11.81015452s) [upload=10233]: Waiting for error group managing segments... Aug 7 19:29:12.220871651 (9h49m11.810216638s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:12.220877713 (9h49m11.810222701s) [upload=10233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:12.220880960 (9h49m11.810225947s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:12.573045808 (9h49m12.162390795s) [upload=10233]: Done uploading segments... Aug 7 19:29:12.573091345 (9h49m12.162436332s) [upload=10233]: Upload finished. err: <nil> Aug 7 19:29:41.735121145 (9h49m41.324466132s) [upload=10244]: Starting upload Aug 7 19:29:41.735168356 (9h49m41.324513363s) [upload=10244]: Done starting upload Aug 7 19:29:41.735255312 (9h49m41.324600289s) [upload=10244]: Uploading segments... Aug 7 19:29:41.735311500 (9h49m41.324656487s) [upload=10244]: Got next segment. Inline: true Aug 7 19:29:41.735314926 (9h49m41.324659913s) [upload=10244]: Waiting for error group managing segments... Aug 7 19:29:41.735368448 (9h49m41.324713435s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:41.735372626 (9h49m41.324717613s) [upload=10244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:41.735374870 (9h49m41.324719858s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:42.000903022 (9h49m41.590248009s) [upload=10244]: Done uploading segments... Aug 7 19:29:42.000921688 (9h49m41.590266675s) [upload=10244]: Upload finished. err: <nil> Aug 7 19:30:09.357577819 (9h50m8.946922806s) [upload=10251]: Starting upload Aug 7 19:30:09.357623787 (9h50m8.946968774s) [upload=10251]: Done starting upload Aug 7 19:30:09.357706075 (9h50m8.947051062s) [upload=10251]: Uploading segments... Aug 7 19:30:09.357828999 (9h50m8.947173986s) [upload=10251]: Got next segment. Inline: true Aug 7 19:30:09.357834209 (9h50m8.947179196s) [upload=10251]: Waiting for error group managing segments... Aug 7 19:30:09.357879085 (9h50m8.947224072s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:09.357883534 (9h50m8.947228521s) [upload=10251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:09.357885748 (9h50m8.947230735s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:09.831093883 (9h50m9.42043887s) [upload=10251]: Done uploading segments... Aug 7 19:30:09.831111727 (9h50m9.420456714s) [upload=10251]: Upload finished. err: <nil> Aug 7 19:30:31.706676511 (9h50m31.296021498s) [upload=10259]: Starting upload Aug 7 19:30:31.706715425 (9h50m31.296060413s) [upload=10259]: Done starting upload Aug 7 19:30:31.706962198 (9h50m31.296307185s) [upload=10259]: Uploading segments... Aug 7 19:30:31.706996403 (9h50m31.29634138s) [upload=10259]: Got next segment. Inline: true Aug 7 19:30:31.707006341 (9h50m31.296351329s) [upload=10259]: Waiting for error group managing segments... Aug 7 19:30:31.707042211 (9h50m31.296387198s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:31.707046168 (9h50m31.296391155s) [upload=10259]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:31.707049004 (9h50m31.296393991s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:32.134768857 (9h50m31.724113844s) [upload=10259]: Done uploading segments... Aug 7 19:30:32.134809155 (9h50m31.724154142s) [upload=10259]: Upload finished. err: <nil> Aug 7 19:31:14.642237281 (9h51m14.231582268s) [upload=10273]: Starting upload Aug 7 19:31:14.642283008 (9h51m14.231627995s) [upload=10273]: Done starting upload Aug 7 19:31:14.642334347 (9h51m14.231679324s) [upload=10273]: Uploading segments... Aug 7 19:31:14.642388370 (9h51m14.231733357s) [upload=10273]: Got next segment. Inline: true Aug 7 19:31:14.642395824 (9h51m14.231740841s) [upload=10273]: Waiting for error group managing segments... Aug 7 19:31:14.642450859 (9h51m14.231795876s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:14.642458204 (9h51m14.231803231s) [upload=10273]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:14.642463464 (9h51m14.231808481s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:14.916538458 (9h51m14.505883465s) [upload=10273]: Done uploading segments... Aug 7 19:31:14.916565079 (9h51m14.505910066s) [upload=10273]: Upload finished. err: <nil> Aug 7 19:31:59.580470078 (9h51m59.169815065s) [upload=10283]: Starting upload Aug 7 19:31:59.580508693 (9h51m59.16985368s) [upload=10283]: Done starting upload Aug 7 19:31:59.580563077 (9h51m59.169908064s) [upload=10283]: Uploading segments... Aug 7 19:31:59.580599707 (9h51m59.169944694s) [upload=10283]: Got next segment. Inline: true Aug 7 19:31:59.580603684 (9h51m59.169948671s) [upload=10283]: Waiting for error group managing segments... Aug 7 19:31:59.580637348 (9h51m59.169982335s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:59.580640916 (9h51m59.169985893s) [upload=10283]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:59.580643240 (9h51m59.169988227s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:59.894471131 (9h51m59.483816118s) [upload=10283]: Done uploading segments... Aug 7 19:31:59.894488253 (9h51m59.48383323s) [upload=10283]: Upload finished. err: <nil> Aug 7 19:32:38.308640915 (9h52m37.897985902s) [upload=10295]: Starting upload Aug 7 19:32:38.308709886 (9h52m37.898054873s) [upload=10295]: Done starting upload Aug 7 19:32:38.308805570 (9h52m37.898150557s) [upload=10295]: Uploading segments... Aug 7 19:32:38.308879861 (9h52m37.898224838s) [upload=10295]: Got next segment. Inline: true Aug 7 19:32:38.308884580 (9h52m37.898229567s) [upload=10295]: Waiting for error group managing segments... Aug 7 19:32:38.308924838 (9h52m37.898269825s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:32:38.308929156 (9h52m37.898274143s) [upload=10295]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:32:38.308931530 (9h52m37.898276517s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:32:38.590415960 (9h52m38.179760957s) [upload=10295]: Done uploading segments... Aug 7 19:32:38.590438433 (9h52m38.179783421s) [upload=10295]: Upload finished. err: <nil> Duration: 1.849697217s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................................X........X..] duration: 1.845452995s FAIL (2025-08-07 19:34:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== m_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Aug 7 19:28:33.238514538 (9h48m32.827859525s) [upload=10221]: Starting upload Aug 7 19:28:33.238562180 (9h48m32.827907157s) [upload=10221]: Done starting upload Aug 7 19:28:33.238627795 (9h48m32.827972782s) [upload=10221]: Uploading segments... Aug 7 19:28:33.238663914 (9h48m32.828008891s) [upload=10221]: Got next segment. Inline: true Aug 7 19:28:33.238667501 (9h48m32.828012488s) [upload=10221]: Waiting for error group managing segments... Aug 7 19:28:33.238704382 (9h48m32.828049429s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:33.238711796 (9h48m32.828056783s) [upload=10221]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:33.238715493 (9h48m32.82806049s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:33.658266417 (9h48m33.247611404s) [upload=10221]: Done uploading segments... Aug 7 19:28:33.658283299 (9h48m33.247628286s) [upload=10221]: Upload finished. err: <nil> Aug 7 19:28:51.361704008 (9h48m50.951048995s) [upload=10225]: Starting upload Aug 7 19:28:51.361752521 (9h48m50.951097508s) [upload=10225]: Done starting upload Aug 7 19:28:51.361814239 (9h48m50.951159226s) [upload=10225]: Uploading segments... Aug 7 19:28:51.361846150 (9h48m50.951191137s) [upload=10225]: Got next segment. Inline: true Aug 7 19:28:51.361849707 (9h48m50.951194695s) [upload=10225]: Waiting for error group managing segments... Aug 7 19:28:51.361880727 (9h48m50.951225714s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:51.361884544 (9h48m50.951229531s) [upload=10225]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:51.361886908 (9h48m50.951231895s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:51.634702903 (9h48m51.22404789s) [upload=10225]: Done uploading segments... Aug 7 19:28:51.634737349 (9h48m51.224082326s) [upload=10225]: Upload finished. err: <nil> Aug 7 19:29:12.220554896 (9h49m11.809899883s) [upload=10233]: Starting upload Aug 7 19:29:12.220627544 (9h49m11.809972551s) [upload=10233]: Done starting upload Aug 7 19:29:12.220708579 (9h49m11.810053576s) [upload=10233]: Uploading segments... Aug 7 19:29:12.220803531 (9h49m11.810148518s) [upload=10233]: Got next segment. Inline: true Aug 7 19:29:12.220809533 (9h49m11.81015452s) [upload=10233]: Waiting for error group managing segments... Aug 7 19:29:12.220871651 (9h49m11.810216638s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:12.220877713 (9h49m11.810222701s) [upload=10233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:12.220880960 (9h49m11.810225947s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:12.573045808 (9h49m12.162390795s) [upload=10233]: Done uploading segments... Aug 7 19:29:12.573091345 (9h49m12.162436332s) [upload=10233]: Upload finished. err: <nil> Aug 7 19:29:41.735121145 (9h49m41.324466132s) [upload=10244]: Starting upload Aug 7 19:29:41.735168356 (9h49m41.324513363s) [upload=10244]: Done starting upload Aug 7 19:29:41.735255312 (9h49m41.324600289s) [upload=10244]: Uploading segments... Aug 7 19:29:41.735311500 (9h49m41.324656487s) [upload=10244]: Got next segment. Inline: true Aug 7 19:29:41.735314926 (9h49m41.324659913s) [upload=10244]: Waiting for error group managing segments... Aug 7 19:29:41.735368448 (9h49m41.324713435s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:41.735372626 (9h49m41.324717613s) [upload=10244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:41.735374870 (9h49m41.324719858s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:42.000903022 (9h49m41.590248009s) [upload=10244]: Done uploading segments... Aug 7 19:29:42.000921688 (9h49m41.590266675s) [upload=10244]: Upload finished. err: <nil> Aug 7 19:30:09.357577819 (9h50m8.946922806s) [upload=10251]: Starting upload Aug 7 19:30:09.357623787 (9h50m8.946968774s) [upload=10251]: Done starting upload Aug 7 19:30:09.357706075 (9h50m8.947051062s) [upload=10251]: Uploading segments... Aug 7 19:30:09.357828999 (9h50m8.947173986s) [upload=10251]: Got next segment. Inline: true Aug 7 19:30:09.357834209 (9h50m8.947179196s) [upload=10251]: Waiting for error group managing segments... Aug 7 19:30:09.357879085 (9h50m8.947224072s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:09.357883534 (9h50m8.947228521s) [upload=10251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:09.357885748 (9h50m8.947230735s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:09.831093883 (9h50m9.42043887s) [upload=10251]: Done uploading segments... Aug 7 19:30:09.831111727 (9h50m9.420456714s) [upload=10251]: Upload finished. err: <nil> Aug 7 19:30:31.706676511 (9h50m31.296021498s) [upload=10259]: Starting upload Aug 7 19:30:31.706715425 (9h50m31.296060413s) [upload=10259]: Done starting upload Aug 7 19:30:31.706962198 (9h50m31.296307185s) [upload=10259]: Uploading segments... Aug 7 19:30:31.706996403 (9h50m31.29634138s) [upload=10259]: Got next segment. Inline: true Aug 7 19:30:31.707006341 (9h50m31.296351329s) [upload=10259]: Waiting for error group managing segments... Aug 7 19:30:31.707042211 (9h50m31.296387198s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:31.707046168 (9h50m31.296391155s) [upload=10259]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:31.707049004 (9h50m31.296393991s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:32.134768857 (9h50m31.724113844s) [upload=10259]: Done uploading segments... Aug 7 19:30:32.134809155 (9h50m31.724154142s) [upload=10259]: Upload finished. err: <nil> Aug 7 19:31:14.642237281 (9h51m14.231582268s) [upload=10273]: Starting upload Aug 7 19:31:14.642283008 (9h51m14.231627995s) [upload=10273]: Done starting upload Aug 7 19:31:14.642334347 (9h51m14.231679324s) [upload=10273]: Uploading segments... Aug 7 19:31:14.642388370 (9h51m14.231733357s) [upload=10273]: Got next segment. Inline: true Aug 7 19:31:14.642395824 (9h51m14.231740841s) [upload=10273]: Waiting for error group managing segments... Aug 7 19:31:14.642450859 (9h51m14.231795876s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:14.642458204 (9h51m14.231803231s) [upload=10273]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:14.642463464 (9h51m14.231808481s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:14.916538458 (9h51m14.505883465s) [upload=10273]: Done uploading segments... Aug 7 19:31:14.916565079 (9h51m14.505910066s) [upload=10273]: Upload finished. err: <nil> Aug 7 19:31:59.580470078 (9h51m59.169815065s) [upload=10283]: Starting upload Aug 7 19:31:59.580508693 (9h51m59.16985368s) [upload=10283]: Done starting upload Aug 7 19:31:59.580563077 (9h51m59.169908064s) [upload=10283]: Uploading segments... Aug 7 19:31:59.580599707 (9h51m59.169944694s) [upload=10283]: Got next segment. Inline: true Aug 7 19:31:59.580603684 (9h51m59.169948671s) [upload=10283]: Waiting for error group managing segments... Aug 7 19:31:59.580637348 (9h51m59.169982335s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:59.580640916 (9h51m59.169985893s) [upload=10283]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:59.580643240 (9h51m59.169988227s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:59.894471131 (9h51m59.483816118s) [upload=10283]: Done uploading segments... Aug 7 19:31:59.894488253 (9h51m59.48383323s) [upload=10283]: Upload finished. err: <nil> Aug 7 19:32:38.308640915 (9h52m37.897985902s) [upload=10295]: Starting upload Aug 7 19:32:38.308709886 (9h52m37.898054873s) [upload=10295]: Done starting upload Aug 7 19:32:38.308805570 (9h52m37.898150557s) [upload=10295]: Uploading segments... Aug 7 19:32:38.308879861 (9h52m37.898224838s) [upload=10295]: Got next segment. Inline: true Aug 7 19:32:38.308884580 (9h52m37.898229567s) [upload=10295]: Waiting for error group managing segments... Aug 7 19:32:38.308924838 (9h52m37.898269825s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:32:38.308929156 (9h52m37.898274143s) [upload=10295]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:32:38.308931530 (9h52m37.898276517s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:32:38.590415960 (9h52m38.179760957s) [upload=10295]: Done uploading segments... Aug 7 19:32:38.590438433 (9h52m38.179783421s) [upload=10295]: Upload finished. err: <nil> Duration: 1.845480659s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X........X......................................] duration: 5.679750731s FAIL (2025-08-07 19:34:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.679813752s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X........X.] duration: 1.840061081s FAIL (2025-08-07 19:33:54 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== m_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Aug 7 19:28:33.238514538 (9h48m32.827859525s) [upload=10221]: Starting upload Aug 7 19:28:33.238562180 (9h48m32.827907157s) [upload=10221]: Done starting upload Aug 7 19:28:33.238627795 (9h48m32.827972782s) [upload=10221]: Uploading segments... Aug 7 19:28:33.238663914 (9h48m32.828008891s) [upload=10221]: Got next segment. Inline: true Aug 7 19:28:33.238667501 (9h48m32.828012488s) [upload=10221]: Waiting for error group managing segments... Aug 7 19:28:33.238704382 (9h48m32.828049429s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:33.238711796 (9h48m32.828056783s) [upload=10221]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:33.238715493 (9h48m32.82806049s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:33.658266417 (9h48m33.247611404s) [upload=10221]: Done uploading segments... Aug 7 19:28:33.658283299 (9h48m33.247628286s) [upload=10221]: Upload finished. err: <nil> Aug 7 19:28:51.361704008 (9h48m50.951048995s) [upload=10225]: Starting upload Aug 7 19:28:51.361752521 (9h48m50.951097508s) [upload=10225]: Done starting upload Aug 7 19:28:51.361814239 (9h48m50.951159226s) [upload=10225]: Uploading segments... Aug 7 19:28:51.361846150 (9h48m50.951191137s) [upload=10225]: Got next segment. Inline: true Aug 7 19:28:51.361849707 (9h48m50.951194695s) [upload=10225]: Waiting for error group managing segments... Aug 7 19:28:51.361880727 (9h48m50.951225714s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:51.361884544 (9h48m50.951229531s) [upload=10225]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:51.361886908 (9h48m50.951231895s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:51.634702903 (9h48m51.22404789s) [upload=10225]: Done uploading segments... Aug 7 19:28:51.634737349 (9h48m51.224082326s) [upload=10225]: Upload finished. err: <nil> Aug 7 19:29:12.220554896 (9h49m11.809899883s) [upload=10233]: Starting upload Aug 7 19:29:12.220627544 (9h49m11.809972551s) [upload=10233]: Done starting upload Aug 7 19:29:12.220708579 (9h49m11.810053576s) [upload=10233]: Uploading segments... Aug 7 19:29:12.220803531 (9h49m11.810148518s) [upload=10233]: Got next segment. Inline: true Aug 7 19:29:12.220809533 (9h49m11.81015452s) [upload=10233]: Waiting for error group managing segments... Aug 7 19:29:12.220871651 (9h49m11.810216638s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:12.220877713 (9h49m11.810222701s) [upload=10233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:12.220880960 (9h49m11.810225947s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:12.573045808 (9h49m12.162390795s) [upload=10233]: Done uploading segments... Aug 7 19:29:12.573091345 (9h49m12.162436332s) [upload=10233]: Upload finished. err: <nil> Aug 7 19:29:41.735121145 (9h49m41.324466132s) [upload=10244]: Starting upload Aug 7 19:29:41.735168356 (9h49m41.324513363s) [upload=10244]: Done starting upload Aug 7 19:29:41.735255312 (9h49m41.324600289s) [upload=10244]: Uploading segments... Aug 7 19:29:41.735311500 (9h49m41.324656487s) [upload=10244]: Got next segment. Inline: true Aug 7 19:29:41.735314926 (9h49m41.324659913s) [upload=10244]: Waiting for error group managing segments... Aug 7 19:29:41.735368448 (9h49m41.324713435s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:41.735372626 (9h49m41.324717613s) [upload=10244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:41.735374870 (9h49m41.324719858s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:42.000903022 (9h49m41.590248009s) [upload=10244]: Done uploading segments... Aug 7 19:29:42.000921688 (9h49m41.590266675s) [upload=10244]: Upload finished. err: <nil> Aug 7 19:30:09.357577819 (9h50m8.946922806s) [upload=10251]: Starting upload Aug 7 19:30:09.357623787 (9h50m8.946968774s) [upload=10251]: Done starting upload Aug 7 19:30:09.357706075 (9h50m8.947051062s) [upload=10251]: Uploading segments... Aug 7 19:30:09.357828999 (9h50m8.947173986s) [upload=10251]: Got next segment. Inline: true Aug 7 19:30:09.357834209 (9h50m8.947179196s) [upload=10251]: Waiting for error group managing segments... Aug 7 19:30:09.357879085 (9h50m8.947224072s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:09.357883534 (9h50m8.947228521s) [upload=10251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:09.357885748 (9h50m8.947230735s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:09.831093883 (9h50m9.42043887s) [upload=10251]: Done uploading segments... Aug 7 19:30:09.831111727 (9h50m9.420456714s) [upload=10251]: Upload finished. err: <nil> Aug 7 19:30:31.706676511 (9h50m31.296021498s) [upload=10259]: Starting upload Aug 7 19:30:31.706715425 (9h50m31.296060413s) [upload=10259]: Done starting upload Aug 7 19:30:31.706962198 (9h50m31.296307185s) [upload=10259]: Uploading segments... Aug 7 19:30:31.706996403 (9h50m31.29634138s) [upload=10259]: Got next segment. Inline: true Aug 7 19:30:31.707006341 (9h50m31.296351329s) [upload=10259]: Waiting for error group managing segments... Aug 7 19:30:31.707042211 (9h50m31.296387198s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:31.707046168 (9h50m31.296391155s) [upload=10259]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:31.707049004 (9h50m31.296393991s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:32.134768857 (9h50m31.724113844s) [upload=10259]: Done uploading segments... Aug 7 19:30:32.134809155 (9h50m31.724154142s) [upload=10259]: Upload finished. err: <nil> Aug 7 19:31:14.642237281 (9h51m14.231582268s) [upload=10273]: Starting upload Aug 7 19:31:14.642283008 (9h51m14.231627995s) [upload=10273]: Done starting upload Aug 7 19:31:14.642334347 (9h51m14.231679324s) [upload=10273]: Uploading segments... Aug 7 19:31:14.642388370 (9h51m14.231733357s) [upload=10273]: Got next segment. Inline: true Aug 7 19:31:14.642395824 (9h51m14.231740841s) [upload=10273]: Waiting for error group managing segments... Aug 7 19:31:14.642450859 (9h51m14.231795876s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:14.642458204 (9h51m14.231803231s) [upload=10273]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:14.642463464 (9h51m14.231808481s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:14.916538458 (9h51m14.505883465s) [upload=10273]: Done uploading segments... Aug 7 19:31:14.916565079 (9h51m14.505910066s) [upload=10273]: Upload finished. err: <nil> Aug 7 19:31:59.580470078 (9h51m59.169815065s) [upload=10283]: Starting upload Aug 7 19:31:59.580508693 (9h51m59.16985368s) [upload=10283]: Done starting upload Aug 7 19:31:59.580563077 (9h51m59.169908064s) [upload=10283]: Uploading segments... Aug 7 19:31:59.580599707 (9h51m59.169944694s) [upload=10283]: Got next segment. Inline: true Aug 7 19:31:59.580603684 (9h51m59.169948671s) [upload=10283]: Waiting for error group managing segments... Aug 7 19:31:59.580637348 (9h51m59.169982335s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:59.580640916 (9h51m59.169985893s) [upload=10283]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:59.580643240 (9h51m59.169988227s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:59.894471131 (9h51m59.483816118s) [upload=10283]: Done uploading segments... Aug 7 19:31:59.894488253 (9h51m59.48383323s) [upload=10283]: Upload finished. err: <nil> Aug 7 19:32:38.308640915 (9h52m37.897985902s) [upload=10295]: Starting upload Aug 7 19:32:38.308709886 (9h52m37.898054873s) [upload=10295]: Done starting upload Aug 7 19:32:38.308805570 (9h52m37.898150557s) [upload=10295]: Uploading segments... Aug 7 19:32:38.308879861 (9h52m37.898224838s) [upload=10295]: Got next segment. Inline: true Aug 7 19:32:38.308884580 (9h52m37.898229567s) [upload=10295]: Waiting for error group managing segments... Aug 7 19:32:38.308924838 (9h52m37.898269825s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:32:38.308929156 (9h52m37.898274143s) [upload=10295]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:32:38.308931530 (9h52m37.898276517s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:32:38.590415960 (9h52m38.179760957s) [upload=10295]: Done uploading segments... Aug 7 19:32:38.590438433 (9h52m38.179783421s) [upload=10295]: Upload finished. err: <nil> Duration: 1.840088625s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X........X.....................................] duration: 6.350437624s FAIL (2025-08-07 19:33:34 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 6.350464285s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X........X] duration: 1.385903307s FAIL (2025-08-07 19:32:39 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== m_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Aug 7 19:28:33.238514538 (9h48m32.827859525s) [upload=10221]: Starting upload Aug 7 19:28:33.238562180 (9h48m32.827907157s) [upload=10221]: Done starting upload Aug 7 19:28:33.238627795 (9h48m32.827972782s) [upload=10221]: Uploading segments... Aug 7 19:28:33.238663914 (9h48m32.828008891s) [upload=10221]: Got next segment. Inline: true Aug 7 19:28:33.238667501 (9h48m32.828012488s) [upload=10221]: Waiting for error group managing segments... Aug 7 19:28:33.238704382 (9h48m32.828049429s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:33.238711796 (9h48m32.828056783s) [upload=10221]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:33.238715493 (9h48m32.82806049s) [upload=10221]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:33.658266417 (9h48m33.247611404s) [upload=10221]: Done uploading segments... Aug 7 19:28:33.658283299 (9h48m33.247628286s) [upload=10221]: Upload finished. err: <nil> Aug 7 19:28:51.361704008 (9h48m50.951048995s) [upload=10225]: Starting upload Aug 7 19:28:51.361752521 (9h48m50.951097508s) [upload=10225]: Done starting upload Aug 7 19:28:51.361814239 (9h48m50.951159226s) [upload=10225]: Uploading segments... Aug 7 19:28:51.361846150 (9h48m50.951191137s) [upload=10225]: Got next segment. Inline: true Aug 7 19:28:51.361849707 (9h48m50.951194695s) [upload=10225]: Waiting for error group managing segments... Aug 7 19:28:51.361880727 (9h48m50.951225714s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:28:51.361884544 (9h48m50.951229531s) [upload=10225]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:28:51.361886908 (9h48m50.951231895s) [upload=10225]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:28:51.634702903 (9h48m51.22404789s) [upload=10225]: Done uploading segments... Aug 7 19:28:51.634737349 (9h48m51.224082326s) [upload=10225]: Upload finished. err: <nil> Aug 7 19:29:12.220554896 (9h49m11.809899883s) [upload=10233]: Starting upload Aug 7 19:29:12.220627544 (9h49m11.809972551s) [upload=10233]: Done starting upload Aug 7 19:29:12.220708579 (9h49m11.810053576s) [upload=10233]: Uploading segments... Aug 7 19:29:12.220803531 (9h49m11.810148518s) [upload=10233]: Got next segment. Inline: true Aug 7 19:29:12.220809533 (9h49m11.81015452s) [upload=10233]: Waiting for error group managing segments... Aug 7 19:29:12.220871651 (9h49m11.810216638s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:12.220877713 (9h49m11.810222701s) [upload=10233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:12.220880960 (9h49m11.810225947s) [upload=10233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:12.573045808 (9h49m12.162390795s) [upload=10233]: Done uploading segments... Aug 7 19:29:12.573091345 (9h49m12.162436332s) [upload=10233]: Upload finished. err: <nil> Aug 7 19:29:41.735121145 (9h49m41.324466132s) [upload=10244]: Starting upload Aug 7 19:29:41.735168356 (9h49m41.324513363s) [upload=10244]: Done starting upload Aug 7 19:29:41.735255312 (9h49m41.324600289s) [upload=10244]: Uploading segments... Aug 7 19:29:41.735311500 (9h49m41.324656487s) [upload=10244]: Got next segment. Inline: true Aug 7 19:29:41.735314926 (9h49m41.324659913s) [upload=10244]: Waiting for error group managing segments... Aug 7 19:29:41.735368448 (9h49m41.324713435s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:29:41.735372626 (9h49m41.324717613s) [upload=10244]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:29:41.735374870 (9h49m41.324719858s) [upload=10244]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:29:42.000903022 (9h49m41.590248009s) [upload=10244]: Done uploading segments... Aug 7 19:29:42.000921688 (9h49m41.590266675s) [upload=10244]: Upload finished. err: <nil> Aug 7 19:30:09.357577819 (9h50m8.946922806s) [upload=10251]: Starting upload Aug 7 19:30:09.357623787 (9h50m8.946968774s) [upload=10251]: Done starting upload Aug 7 19:30:09.357706075 (9h50m8.947051062s) [upload=10251]: Uploading segments... Aug 7 19:30:09.357828999 (9h50m8.947173986s) [upload=10251]: Got next segment. Inline: true Aug 7 19:30:09.357834209 (9h50m8.947179196s) [upload=10251]: Waiting for error group managing segments... Aug 7 19:30:09.357879085 (9h50m8.947224072s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:09.357883534 (9h50m8.947228521s) [upload=10251]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:09.357885748 (9h50m8.947230735s) [upload=10251]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:09.831093883 (9h50m9.42043887s) [upload=10251]: Done uploading segments... Aug 7 19:30:09.831111727 (9h50m9.420456714s) [upload=10251]: Upload finished. err: <nil> Aug 7 19:30:31.706676511 (9h50m31.296021498s) [upload=10259]: Starting upload Aug 7 19:30:31.706715425 (9h50m31.296060413s) [upload=10259]: Done starting upload Aug 7 19:30:31.706962198 (9h50m31.296307185s) [upload=10259]: Uploading segments... Aug 7 19:30:31.706996403 (9h50m31.29634138s) [upload=10259]: Got next segment. Inline: true Aug 7 19:30:31.707006341 (9h50m31.296351329s) [upload=10259]: Waiting for error group managing segments... Aug 7 19:30:31.707042211 (9h50m31.296387198s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:30:31.707046168 (9h50m31.296391155s) [upload=10259]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:30:31.707049004 (9h50m31.296393991s) [upload=10259]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:30:32.134768857 (9h50m31.724113844s) [upload=10259]: Done uploading segments... Aug 7 19:30:32.134809155 (9h50m31.724154142s) [upload=10259]: Upload finished. err: <nil> Aug 7 19:31:14.642237281 (9h51m14.231582268s) [upload=10273]: Starting upload Aug 7 19:31:14.642283008 (9h51m14.231627995s) [upload=10273]: Done starting upload Aug 7 19:31:14.642334347 (9h51m14.231679324s) [upload=10273]: Uploading segments... Aug 7 19:31:14.642388370 (9h51m14.231733357s) [upload=10273]: Got next segment. Inline: true Aug 7 19:31:14.642395824 (9h51m14.231740841s) [upload=10273]: Waiting for error group managing segments... Aug 7 19:31:14.642450859 (9h51m14.231795876s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:14.642458204 (9h51m14.231803231s) [upload=10273]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:14.642463464 (9h51m14.231808481s) [upload=10273]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:14.916538458 (9h51m14.505883465s) [upload=10273]: Done uploading segments... Aug 7 19:31:14.916565079 (9h51m14.505910066s) [upload=10273]: Upload finished. err: <nil> Aug 7 19:31:59.580470078 (9h51m59.169815065s) [upload=10283]: Starting upload Aug 7 19:31:59.580508693 (9h51m59.16985368s) [upload=10283]: Done starting upload Aug 7 19:31:59.580563077 (9h51m59.169908064s) [upload=10283]: Uploading segments... Aug 7 19:31:59.580599707 (9h51m59.169944694s) [upload=10283]: Got next segment. Inline: true Aug 7 19:31:59.580603684 (9h51m59.169948671s) [upload=10283]: Waiting for error group managing segments... Aug 7 19:31:59.580637348 (9h51m59.169982335s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:31:59.580640916 (9h51m59.169985893s) [upload=10283]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:31:59.580643240 (9h51m59.169988227s) [upload=10283]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:31:59.894471131 (9h51m59.483816118s) [upload=10283]: Done uploading segments... Aug 7 19:31:59.894488253 (9h51m59.48383323s) [upload=10283]: Upload finished. err: <nil> Aug 7 19:32:38.308640915 (9h52m37.897985902s) [upload=10295]: Starting upload Aug 7 19:32:38.308709886 (9h52m37.898054873s) [upload=10295]: Done starting upload Aug 7 19:32:38.308805570 (9h52m37.898150557s) [upload=10295]: Uploading segments... Aug 7 19:32:38.308879861 (9h52m37.898224838s) [upload=10295]: Got next segment. Inline: true Aug 7 19:32:38.308884580 (9h52m37.898229567s) [upload=10295]: Waiting for error group managing segments... Aug 7 19:32:38.308924838 (9h52m37.898269825s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:32:38.308929156 (9h52m37.898274143s) [upload=10295]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:32:38.308931530 (9h52m37.898276517s) [upload=10295]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:32:38.590415960 (9h52m38.179760957s) [upload=10295]: Done uploading segments... Aug 7 19:32:38.590438433 (9h52m38.179783421s) [upload=10295]: Upload finished. err: <nil> Duration: 1.385923916s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X........X....................................] duration: 5.997718102s FAIL (2025-08-07 19:32:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.997745545s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.070340254s FAIL (2025-08-07 19:32:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== b.BatchRequestItem_ObjectCommit Aug 7 19:19:25.172538463 (9h39m24.76188345s) [upload=10067]: Done uploading segments... Aug 7 19:19:25.172556097 (9h39m24.761901084s) [upload=10067]: Upload finished. err: <nil> Aug 7 19:19:59.108396078 (9h39m58.697741065s) [upload=10075]: Starting upload Aug 7 19:19:59.108438248 (9h39m58.697783225s) [upload=10075]: Done starting upload Aug 7 19:19:59.108486270 (9h39m58.697831257s) [upload=10075]: Uploading segments... Aug 7 19:19:59.108519834 (9h39m58.697864821s) [upload=10075]: Got next segment. Inline: true Aug 7 19:19:59.108523431 (9h39m58.697868418s) [upload=10075]: Waiting for error group managing segments... Aug 7 19:19:59.108555602 (9h39m58.697900589s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:19:59.108560452 (9h39m58.697905439s) [upload=10075]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:19:59.108563317 (9h39m58.697908305s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:19:59.535030122 (9h39m59.124375099s) [upload=10075]: Done uploading segments... Aug 7 19:19:59.535046735 (9h39m59.124391722s) [upload=10075]: Upload finished. err: <nil> Aug 7 19:21:07.972243470 (9h41m7.561588457s) [upload=10091]: Starting upload Aug 7 19:21:07.972303556 (9h41m7.561648543s) [upload=10091]: Done starting upload Aug 7 19:21:07.972352188 (9h41m7.561697175s) [upload=10091]: Uploading segments... Aug 7 19:21:07.972389430 (9h41m7.561734417s) [upload=10091]: Got next segment. Inline: true Aug 7 19:21:07.972393137 (9h41m7.561738124s) [upload=10091]: Waiting for error group managing segments... Aug 7 19:21:07.972440648 (9h41m7.561785635s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:21:07.972445257 (9h41m7.561790244s) [upload=10091]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:21:07.972447651 (9h41m7.561792639s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:21:08.256090200 (9h41m7.845435187s) [upload=10091]: Done uploading segments... Aug 7 19:21:08.256108215 (9h41m7.845453202s) [upload=10091]: Upload finished. err: <nil> Aug 7 19:22:04.622194388 (9h42m4.211539375s) [upload=10111]: Starting upload Aug 7 19:22:04.622243893 (9h42m4.21158888s) [upload=10111]: Done starting upload Aug 7 19:22:04.622314057 (9h42m4.211659044s) [upload=10111]: Uploading segments... Aug 7 19:22:04.622362129 (9h42m4.211707156s) [upload=10111]: Got next segment. Inline: true Aug 7 19:22:04.622368521 (9h42m4.211713528s) [upload=10111]: Waiting for error group managing segments... Aug 7 19:22:04.622413116 (9h42m4.211758104s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:22:04.622417394 (9h42m4.211762382s) [upload=10111]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:22:04.622419959 (9h42m4.211764946s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:22:04.930824935 (9h42m4.520169922s) [upload=10111]: Done uploading segments... Aug 7 19:22:04.930844082 (9h42m4.520189069s) [upload=10111]: Upload finished. err: <nil> Aug 7 19:23:18.125278170 (9h43m17.714623157s) [upload=10129]: Starting upload Aug 7 19:23:18.125322425 (9h43m17.714667412s) [upload=10129]: Done starting upload Aug 7 19:23:18.125398790 (9h43m17.714743768s) [upload=10129]: Uploading segments... Aug 7 19:23:18.125427335 (9h43m17.714772322s) [upload=10129]: Got next segment. Inline: true Aug 7 19:23:18.125430982 (9h43m17.714775969s) [upload=10129]: Waiting for error group managing segments... Aug 7 19:23:18.125486578 (9h43m17.714831565s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:23:18.125491038 (9h43m17.714836025s) [upload=10129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:23:18.125493492 (9h43m17.714838479s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:23:18.413472808 (9h43m18.002817795s) [upload=10129]: Done uploading segments... Aug 7 19:23:18.413488809 (9h43m18.002833786s) [upload=10129]: Upload finished. err: <nil> Aug 7 19:24:20.169825506 (9h44m19.759170483s) [upload=10145]: Starting upload Aug 7 19:24:20.169871795 (9h44m19.759216772s) [upload=10145]: Done starting upload Aug 7 19:24:20.169923194 (9h44m19.759268171s) [upload=10145]: Uploading segments... Aug 7 19:24:20.169954353 (9h44m19.75929934s) [upload=10145]: Got next segment. Inline: true Aug 7 19:24:20.169958020 (9h44m19.759303007s) [upload=10145]: Waiting for error group managing segments... Aug 7 19:24:20.169989220 (9h44m19.759334197s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:20.169993168 (9h44m19.759338155s) [upload=10145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:20.169995552 (9h44m19.759340539s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:20.664326400 (9h44m20.253671387s) [upload=10145]: Done uploading segments... Aug 7 19:24:20.664347220 (9h44m20.253692207s) [upload=10145]: Upload finished. err: <nil> Aug 7 19:24:57.194307899 (9h44m56.783652886s) [upload=10158]: Starting upload Aug 7 19:24:57.194352165 (9h44m56.783697142s) [upload=10158]: Done starting upload Aug 7 19:24:57.194396358 (9h44m56.783741346s) [upload=10158]: Uploading segments... Aug 7 19:24:57.194425755 (9h44m56.783770742s) [upload=10158]: Got next segment. Inline: true Aug 7 19:24:57.194429002 (9h44m56.783773989s) [upload=10158]: Waiting for error group managing segments... Aug 7 19:24:57.194459169 (9h44m56.783804156s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:57.194462526 (9h44m56.783807513s) [upload=10158]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:57.194464960 (9h44m56.783809947s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:57.508722153 (9h44m57.09806714s) [upload=10158]: Done uploading segments... Aug 7 19:24:57.508738245 (9h44m57.098083222s) [upload=10158]: Upload finished. err: <nil> Aug 7 19:25:23.090976846 (9h45m22.680321833s) [upload=10165]: Starting upload Aug 7 19:25:23.091016031 (9h45m22.680361018s) [upload=10165]: Done starting upload Aug 7 19:25:23.091078450 (9h45m22.680423427s) [upload=10165]: Uploading segments... Aug 7 19:25:23.091118557 (9h45m22.680463544s) [upload=10165]: Got next segment. Inline: true Aug 7 19:25:23.091122875 (9h45m22.680467852s) [upload=10165]: Waiting for error group managing segments... Aug 7 19:25:23.091155227 (9h45m22.680500214s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:25:23.091158914 (9h45m22.680503901s) [upload=10165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:25:23.091161098 (9h45m22.680506075s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:25:23.354144091 (9h45m22.943489078s) [upload=10165]: Done uploading segments... Aug 7 19:25:23.354161835 (9h45m22.943506822s) [upload=10165]: Upload finished. err: <nil> Aug 7 19:26:33.340151800 (9h46m32.929496787s) [upload=10181]: Starting upload Aug 7 19:26:33.340236143 (9h46m32.92958118s) [upload=10181]: Done starting upload Aug 7 19:26:33.340332376 (9h46m32.929677353s) [upload=10181]: Uploading segments... Aug 7 19:26:33.340373115 (9h46m32.929718112s) [upload=10181]: Got next segment. Inline: true Aug 7 19:26:33.340377673 (9h46m32.92972266s) [upload=10181]: Waiting for error group managing segments... Aug 7 19:26:33.340410516 (9h46m32.929755503s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:26:33.340414664 (9h46m32.929759651s) [upload=10181]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:26:33.340416918 (9h46m32.929761905s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:26:33.616120797 (9h46m33.205465824s) [upload=10181]: Done uploading segments... Aug 7 19:26:33.616142689 (9h46m33.205487666s) [upload=10181]: Upload finished. err: <nil> Aug 7 19:27:09.886832212 (9h47m9.476177199s) [upload=10193]: Starting upload Aug 7 19:27:09.886900242 (9h47m9.476245229s) [upload=10193]: Done starting upload Aug 7 19:27:09.886966249 (9h47m9.476311236s) [upload=10193]: Uploading segments... Aug 7 19:27:09.887030491 (9h47m9.476375478s) [upload=10193]: Got next segment. Inline: true Aug 7 19:27:09.887034268 (9h47m9.476379255s) [upload=10193]: Waiting for error group managing segments... Aug 7 19:27:09.887069426 (9h47m9.476414413s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:09.887098522 (9h47m9.476443509s) [upload=10193]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:09.887100906 (9h47m9.476445893s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Duration: 2.070366916s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X........X...................................] duration: 5.786512828s FAIL (2025-08-07 19:31:27 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.786540229s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.01712407s FAIL (2025-08-07 19:31:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== b.BatchRequestItem_ObjectCommit Aug 7 19:19:25.172538463 (9h39m24.76188345s) [upload=10067]: Done uploading segments... Aug 7 19:19:25.172556097 (9h39m24.761901084s) [upload=10067]: Upload finished. err: <nil> Aug 7 19:19:59.108396078 (9h39m58.697741065s) [upload=10075]: Starting upload Aug 7 19:19:59.108438248 (9h39m58.697783225s) [upload=10075]: Done starting upload Aug 7 19:19:59.108486270 (9h39m58.697831257s) [upload=10075]: Uploading segments... Aug 7 19:19:59.108519834 (9h39m58.697864821s) [upload=10075]: Got next segment. Inline: true Aug 7 19:19:59.108523431 (9h39m58.697868418s) [upload=10075]: Waiting for error group managing segments... Aug 7 19:19:59.108555602 (9h39m58.697900589s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:19:59.108560452 (9h39m58.697905439s) [upload=10075]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:19:59.108563317 (9h39m58.697908305s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:19:59.535030122 (9h39m59.124375099s) [upload=10075]: Done uploading segments... Aug 7 19:19:59.535046735 (9h39m59.124391722s) [upload=10075]: Upload finished. err: <nil> Aug 7 19:21:07.972243470 (9h41m7.561588457s) [upload=10091]: Starting upload Aug 7 19:21:07.972303556 (9h41m7.561648543s) [upload=10091]: Done starting upload Aug 7 19:21:07.972352188 (9h41m7.561697175s) [upload=10091]: Uploading segments... Aug 7 19:21:07.972389430 (9h41m7.561734417s) [upload=10091]: Got next segment. Inline: true Aug 7 19:21:07.972393137 (9h41m7.561738124s) [upload=10091]: Waiting for error group managing segments... Aug 7 19:21:07.972440648 (9h41m7.561785635s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:21:07.972445257 (9h41m7.561790244s) [upload=10091]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:21:07.972447651 (9h41m7.561792639s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:21:08.256090200 (9h41m7.845435187s) [upload=10091]: Done uploading segments... Aug 7 19:21:08.256108215 (9h41m7.845453202s) [upload=10091]: Upload finished. err: <nil> Aug 7 19:22:04.622194388 (9h42m4.211539375s) [upload=10111]: Starting upload Aug 7 19:22:04.622243893 (9h42m4.21158888s) [upload=10111]: Done starting upload Aug 7 19:22:04.622314057 (9h42m4.211659044s) [upload=10111]: Uploading segments... Aug 7 19:22:04.622362129 (9h42m4.211707156s) [upload=10111]: Got next segment. Inline: true Aug 7 19:22:04.622368521 (9h42m4.211713528s) [upload=10111]: Waiting for error group managing segments... Aug 7 19:22:04.622413116 (9h42m4.211758104s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:22:04.622417394 (9h42m4.211762382s) [upload=10111]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:22:04.622419959 (9h42m4.211764946s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:22:04.930824935 (9h42m4.520169922s) [upload=10111]: Done uploading segments... Aug 7 19:22:04.930844082 (9h42m4.520189069s) [upload=10111]: Upload finished. err: <nil> Aug 7 19:23:18.125278170 (9h43m17.714623157s) [upload=10129]: Starting upload Aug 7 19:23:18.125322425 (9h43m17.714667412s) [upload=10129]: Done starting upload Aug 7 19:23:18.125398790 (9h43m17.714743768s) [upload=10129]: Uploading segments... Aug 7 19:23:18.125427335 (9h43m17.714772322s) [upload=10129]: Got next segment. Inline: true Aug 7 19:23:18.125430982 (9h43m17.714775969s) [upload=10129]: Waiting for error group managing segments... Aug 7 19:23:18.125486578 (9h43m17.714831565s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:23:18.125491038 (9h43m17.714836025s) [upload=10129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:23:18.125493492 (9h43m17.714838479s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:23:18.413472808 (9h43m18.002817795s) [upload=10129]: Done uploading segments... Aug 7 19:23:18.413488809 (9h43m18.002833786s) [upload=10129]: Upload finished. err: <nil> Aug 7 19:24:20.169825506 (9h44m19.759170483s) [upload=10145]: Starting upload Aug 7 19:24:20.169871795 (9h44m19.759216772s) [upload=10145]: Done starting upload Aug 7 19:24:20.169923194 (9h44m19.759268171s) [upload=10145]: Uploading segments... Aug 7 19:24:20.169954353 (9h44m19.75929934s) [upload=10145]: Got next segment. Inline: true Aug 7 19:24:20.169958020 (9h44m19.759303007s) [upload=10145]: Waiting for error group managing segments... Aug 7 19:24:20.169989220 (9h44m19.759334197s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:20.169993168 (9h44m19.759338155s) [upload=10145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:20.169995552 (9h44m19.759340539s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:20.664326400 (9h44m20.253671387s) [upload=10145]: Done uploading segments... Aug 7 19:24:20.664347220 (9h44m20.253692207s) [upload=10145]: Upload finished. err: <nil> Aug 7 19:24:57.194307899 (9h44m56.783652886s) [upload=10158]: Starting upload Aug 7 19:24:57.194352165 (9h44m56.783697142s) [upload=10158]: Done starting upload Aug 7 19:24:57.194396358 (9h44m56.783741346s) [upload=10158]: Uploading segments... Aug 7 19:24:57.194425755 (9h44m56.783770742s) [upload=10158]: Got next segment. Inline: true Aug 7 19:24:57.194429002 (9h44m56.783773989s) [upload=10158]: Waiting for error group managing segments... Aug 7 19:24:57.194459169 (9h44m56.783804156s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:57.194462526 (9h44m56.783807513s) [upload=10158]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:57.194464960 (9h44m56.783809947s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:57.508722153 (9h44m57.09806714s) [upload=10158]: Done uploading segments... Aug 7 19:24:57.508738245 (9h44m57.098083222s) [upload=10158]: Upload finished. err: <nil> Aug 7 19:25:23.090976846 (9h45m22.680321833s) [upload=10165]: Starting upload Aug 7 19:25:23.091016031 (9h45m22.680361018s) [upload=10165]: Done starting upload Aug 7 19:25:23.091078450 (9h45m22.680423427s) [upload=10165]: Uploading segments... Aug 7 19:25:23.091118557 (9h45m22.680463544s) [upload=10165]: Got next segment. Inline: true Aug 7 19:25:23.091122875 (9h45m22.680467852s) [upload=10165]: Waiting for error group managing segments... Aug 7 19:25:23.091155227 (9h45m22.680500214s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:25:23.091158914 (9h45m22.680503901s) [upload=10165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:25:23.091161098 (9h45m22.680506075s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:25:23.354144091 (9h45m22.943489078s) [upload=10165]: Done uploading segments... Aug 7 19:25:23.354161835 (9h45m22.943506822s) [upload=10165]: Upload finished. err: <nil> Aug 7 19:26:33.340151800 (9h46m32.929496787s) [upload=10181]: Starting upload Aug 7 19:26:33.340236143 (9h46m32.92958118s) [upload=10181]: Done starting upload Aug 7 19:26:33.340332376 (9h46m32.929677353s) [upload=10181]: Uploading segments... Aug 7 19:26:33.340373115 (9h46m32.929718112s) [upload=10181]: Got next segment. Inline: true Aug 7 19:26:33.340377673 (9h46m32.92972266s) [upload=10181]: Waiting for error group managing segments... Aug 7 19:26:33.340410516 (9h46m32.929755503s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:26:33.340414664 (9h46m32.929759651s) [upload=10181]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:26:33.340416918 (9h46m32.929761905s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:26:33.616120797 (9h46m33.205465824s) [upload=10181]: Done uploading segments... Aug 7 19:26:33.616142689 (9h46m33.205487666s) [upload=10181]: Upload finished. err: <nil> Aug 7 19:27:09.886832212 (9h47m9.476177199s) [upload=10193]: Starting upload Aug 7 19:27:09.886900242 (9h47m9.476245229s) [upload=10193]: Done starting upload Aug 7 19:27:09.886966249 (9h47m9.476311236s) [upload=10193]: Uploading segments... Aug 7 19:27:09.887030491 (9h47m9.476375478s) [upload=10193]: Got next segment. Inline: true Aug 7 19:27:09.887034268 (9h47m9.476379255s) [upload=10193]: Waiting for error group managing segments... Aug 7 19:27:09.887069426 (9h47m9.476414413s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:09.887098522 (9h47m9.476443509s) [upload=10193]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:09.887100906 (9h47m9.476445893s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Duration: 2.01715082s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X........X..................................] duration: 6.372431076s FAIL (2025-08-07 19:30:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 6.372449882s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.074325608s FAIL (2025-08-07 19:30:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== b.BatchRequestItem_ObjectCommit Aug 7 19:19:25.172538463 (9h39m24.76188345s) [upload=10067]: Done uploading segments... Aug 7 19:19:25.172556097 (9h39m24.761901084s) [upload=10067]: Upload finished. err: <nil> Aug 7 19:19:59.108396078 (9h39m58.697741065s) [upload=10075]: Starting upload Aug 7 19:19:59.108438248 (9h39m58.697783225s) [upload=10075]: Done starting upload Aug 7 19:19:59.108486270 (9h39m58.697831257s) [upload=10075]: Uploading segments... Aug 7 19:19:59.108519834 (9h39m58.697864821s) [upload=10075]: Got next segment. Inline: true Aug 7 19:19:59.108523431 (9h39m58.697868418s) [upload=10075]: Waiting for error group managing segments... Aug 7 19:19:59.108555602 (9h39m58.697900589s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:19:59.108560452 (9h39m58.697905439s) [upload=10075]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:19:59.108563317 (9h39m58.697908305s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:19:59.535030122 (9h39m59.124375099s) [upload=10075]: Done uploading segments... Aug 7 19:19:59.535046735 (9h39m59.124391722s) [upload=10075]: Upload finished. err: <nil> Aug 7 19:21:07.972243470 (9h41m7.561588457s) [upload=10091]: Starting upload Aug 7 19:21:07.972303556 (9h41m7.561648543s) [upload=10091]: Done starting upload Aug 7 19:21:07.972352188 (9h41m7.561697175s) [upload=10091]: Uploading segments... Aug 7 19:21:07.972389430 (9h41m7.561734417s) [upload=10091]: Got next segment. Inline: true Aug 7 19:21:07.972393137 (9h41m7.561738124s) [upload=10091]: Waiting for error group managing segments... Aug 7 19:21:07.972440648 (9h41m7.561785635s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:21:07.972445257 (9h41m7.561790244s) [upload=10091]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:21:07.972447651 (9h41m7.561792639s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:21:08.256090200 (9h41m7.845435187s) [upload=10091]: Done uploading segments... Aug 7 19:21:08.256108215 (9h41m7.845453202s) [upload=10091]: Upload finished. err: <nil> Aug 7 19:22:04.622194388 (9h42m4.211539375s) [upload=10111]: Starting upload Aug 7 19:22:04.622243893 (9h42m4.21158888s) [upload=10111]: Done starting upload Aug 7 19:22:04.622314057 (9h42m4.211659044s) [upload=10111]: Uploading segments... Aug 7 19:22:04.622362129 (9h42m4.211707156s) [upload=10111]: Got next segment. Inline: true Aug 7 19:22:04.622368521 (9h42m4.211713528s) [upload=10111]: Waiting for error group managing segments... Aug 7 19:22:04.622413116 (9h42m4.211758104s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:22:04.622417394 (9h42m4.211762382s) [upload=10111]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:22:04.622419959 (9h42m4.211764946s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:22:04.930824935 (9h42m4.520169922s) [upload=10111]: Done uploading segments... Aug 7 19:22:04.930844082 (9h42m4.520189069s) [upload=10111]: Upload finished. err: <nil> Aug 7 19:23:18.125278170 (9h43m17.714623157s) [upload=10129]: Starting upload Aug 7 19:23:18.125322425 (9h43m17.714667412s) [upload=10129]: Done starting upload Aug 7 19:23:18.125398790 (9h43m17.714743768s) [upload=10129]: Uploading segments... Aug 7 19:23:18.125427335 (9h43m17.714772322s) [upload=10129]: Got next segment. Inline: true Aug 7 19:23:18.125430982 (9h43m17.714775969s) [upload=10129]: Waiting for error group managing segments... Aug 7 19:23:18.125486578 (9h43m17.714831565s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:23:18.125491038 (9h43m17.714836025s) [upload=10129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:23:18.125493492 (9h43m17.714838479s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:23:18.413472808 (9h43m18.002817795s) [upload=10129]: Done uploading segments... Aug 7 19:23:18.413488809 (9h43m18.002833786s) [upload=10129]: Upload finished. err: <nil> Aug 7 19:24:20.169825506 (9h44m19.759170483s) [upload=10145]: Starting upload Aug 7 19:24:20.169871795 (9h44m19.759216772s) [upload=10145]: Done starting upload Aug 7 19:24:20.169923194 (9h44m19.759268171s) [upload=10145]: Uploading segments... Aug 7 19:24:20.169954353 (9h44m19.75929934s) [upload=10145]: Got next segment. Inline: true Aug 7 19:24:20.169958020 (9h44m19.759303007s) [upload=10145]: Waiting for error group managing segments... Aug 7 19:24:20.169989220 (9h44m19.759334197s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:20.169993168 (9h44m19.759338155s) [upload=10145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:20.169995552 (9h44m19.759340539s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:20.664326400 (9h44m20.253671387s) [upload=10145]: Done uploading segments... Aug 7 19:24:20.664347220 (9h44m20.253692207s) [upload=10145]: Upload finished. err: <nil> Aug 7 19:24:57.194307899 (9h44m56.783652886s) [upload=10158]: Starting upload Aug 7 19:24:57.194352165 (9h44m56.783697142s) [upload=10158]: Done starting upload Aug 7 19:24:57.194396358 (9h44m56.783741346s) [upload=10158]: Uploading segments... Aug 7 19:24:57.194425755 (9h44m56.783770742s) [upload=10158]: Got next segment. Inline: true Aug 7 19:24:57.194429002 (9h44m56.783773989s) [upload=10158]: Waiting for error group managing segments... Aug 7 19:24:57.194459169 (9h44m56.783804156s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:57.194462526 (9h44m56.783807513s) [upload=10158]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:57.194464960 (9h44m56.783809947s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:57.508722153 (9h44m57.09806714s) [upload=10158]: Done uploading segments... Aug 7 19:24:57.508738245 (9h44m57.098083222s) [upload=10158]: Upload finished. err: <nil> Aug 7 19:25:23.090976846 (9h45m22.680321833s) [upload=10165]: Starting upload Aug 7 19:25:23.091016031 (9h45m22.680361018s) [upload=10165]: Done starting upload Aug 7 19:25:23.091078450 (9h45m22.680423427s) [upload=10165]: Uploading segments... Aug 7 19:25:23.091118557 (9h45m22.680463544s) [upload=10165]: Got next segment. Inline: true Aug 7 19:25:23.091122875 (9h45m22.680467852s) [upload=10165]: Waiting for error group managing segments... Aug 7 19:25:23.091155227 (9h45m22.680500214s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:25:23.091158914 (9h45m22.680503901s) [upload=10165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:25:23.091161098 (9h45m22.680506075s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:25:23.354144091 (9h45m22.943489078s) [upload=10165]: Done uploading segments... Aug 7 19:25:23.354161835 (9h45m22.943506822s) [upload=10165]: Upload finished. err: <nil> Aug 7 19:26:33.340151800 (9h46m32.929496787s) [upload=10181]: Starting upload Aug 7 19:26:33.340236143 (9h46m32.92958118s) [upload=10181]: Done starting upload Aug 7 19:26:33.340332376 (9h46m32.929677353s) [upload=10181]: Uploading segments... Aug 7 19:26:33.340373115 (9h46m32.929718112s) [upload=10181]: Got next segment. Inline: true Aug 7 19:26:33.340377673 (9h46m32.92972266s) [upload=10181]: Waiting for error group managing segments... Aug 7 19:26:33.340410516 (9h46m32.929755503s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:26:33.340414664 (9h46m32.929759651s) [upload=10181]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:26:33.340416918 (9h46m32.929761905s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:26:33.616120797 (9h46m33.205465824s) [upload=10181]: Done uploading segments... Aug 7 19:26:33.616142689 (9h46m33.205487666s) [upload=10181]: Upload finished. err: <nil> Aug 7 19:27:09.886832212 (9h47m9.476177199s) [upload=10193]: Starting upload Aug 7 19:27:09.886900242 (9h47m9.476245229s) [upload=10193]: Done starting upload Aug 7 19:27:09.886966249 (9h47m9.476311236s) [upload=10193]: Uploading segments... Aug 7 19:27:09.887030491 (9h47m9.476375478s) [upload=10193]: Got next segment. Inline: true Aug 7 19:27:09.887034268 (9h47m9.476379255s) [upload=10193]: Waiting for error group managing segments... Aug 7 19:27:09.887069426 (9h47m9.476414413s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:09.887098522 (9h47m9.476443509s) [upload=10193]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:09.887100906 (9h47m9.476445893s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Duration: 2.07435263s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X........X.................................] duration: 5.83279261s FAIL (2025-08-07 19:30:26 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.83281368s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.162307632s FAIL (2025-08-07 19:30:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== b.BatchRequestItem_ObjectCommit Aug 7 19:19:25.172538463 (9h39m24.76188345s) [upload=10067]: Done uploading segments... Aug 7 19:19:25.172556097 (9h39m24.761901084s) [upload=10067]: Upload finished. err: <nil> Aug 7 19:19:59.108396078 (9h39m58.697741065s) [upload=10075]: Starting upload Aug 7 19:19:59.108438248 (9h39m58.697783225s) [upload=10075]: Done starting upload Aug 7 19:19:59.108486270 (9h39m58.697831257s) [upload=10075]: Uploading segments... Aug 7 19:19:59.108519834 (9h39m58.697864821s) [upload=10075]: Got next segment. Inline: true Aug 7 19:19:59.108523431 (9h39m58.697868418s) [upload=10075]: Waiting for error group managing segments... Aug 7 19:19:59.108555602 (9h39m58.697900589s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:19:59.108560452 (9h39m58.697905439s) [upload=10075]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:19:59.108563317 (9h39m58.697908305s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:19:59.535030122 (9h39m59.124375099s) [upload=10075]: Done uploading segments... Aug 7 19:19:59.535046735 (9h39m59.124391722s) [upload=10075]: Upload finished. err: <nil> Aug 7 19:21:07.972243470 (9h41m7.561588457s) [upload=10091]: Starting upload Aug 7 19:21:07.972303556 (9h41m7.561648543s) [upload=10091]: Done starting upload Aug 7 19:21:07.972352188 (9h41m7.561697175s) [upload=10091]: Uploading segments... Aug 7 19:21:07.972389430 (9h41m7.561734417s) [upload=10091]: Got next segment. Inline: true Aug 7 19:21:07.972393137 (9h41m7.561738124s) [upload=10091]: Waiting for error group managing segments... Aug 7 19:21:07.972440648 (9h41m7.561785635s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:21:07.972445257 (9h41m7.561790244s) [upload=10091]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:21:07.972447651 (9h41m7.561792639s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:21:08.256090200 (9h41m7.845435187s) [upload=10091]: Done uploading segments... Aug 7 19:21:08.256108215 (9h41m7.845453202s) [upload=10091]: Upload finished. err: <nil> Aug 7 19:22:04.622194388 (9h42m4.211539375s) [upload=10111]: Starting upload Aug 7 19:22:04.622243893 (9h42m4.21158888s) [upload=10111]: Done starting upload Aug 7 19:22:04.622314057 (9h42m4.211659044s) [upload=10111]: Uploading segments... Aug 7 19:22:04.622362129 (9h42m4.211707156s) [upload=10111]: Got next segment. Inline: true Aug 7 19:22:04.622368521 (9h42m4.211713528s) [upload=10111]: Waiting for error group managing segments... Aug 7 19:22:04.622413116 (9h42m4.211758104s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:22:04.622417394 (9h42m4.211762382s) [upload=10111]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:22:04.622419959 (9h42m4.211764946s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:22:04.930824935 (9h42m4.520169922s) [upload=10111]: Done uploading segments... Aug 7 19:22:04.930844082 (9h42m4.520189069s) [upload=10111]: Upload finished. err: <nil> Aug 7 19:23:18.125278170 (9h43m17.714623157s) [upload=10129]: Starting upload Aug 7 19:23:18.125322425 (9h43m17.714667412s) [upload=10129]: Done starting upload Aug 7 19:23:18.125398790 (9h43m17.714743768s) [upload=10129]: Uploading segments... Aug 7 19:23:18.125427335 (9h43m17.714772322s) [upload=10129]: Got next segment. Inline: true Aug 7 19:23:18.125430982 (9h43m17.714775969s) [upload=10129]: Waiting for error group managing segments... Aug 7 19:23:18.125486578 (9h43m17.714831565s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:23:18.125491038 (9h43m17.714836025s) [upload=10129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:23:18.125493492 (9h43m17.714838479s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:23:18.413472808 (9h43m18.002817795s) [upload=10129]: Done uploading segments... Aug 7 19:23:18.413488809 (9h43m18.002833786s) [upload=10129]: Upload finished. err: <nil> Aug 7 19:24:20.169825506 (9h44m19.759170483s) [upload=10145]: Starting upload Aug 7 19:24:20.169871795 (9h44m19.759216772s) [upload=10145]: Done starting upload Aug 7 19:24:20.169923194 (9h44m19.759268171s) [upload=10145]: Uploading segments... Aug 7 19:24:20.169954353 (9h44m19.75929934s) [upload=10145]: Got next segment. Inline: true Aug 7 19:24:20.169958020 (9h44m19.759303007s) [upload=10145]: Waiting for error group managing segments... Aug 7 19:24:20.169989220 (9h44m19.759334197s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:20.169993168 (9h44m19.759338155s) [upload=10145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:20.169995552 (9h44m19.759340539s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:20.664326400 (9h44m20.253671387s) [upload=10145]: Done uploading segments... Aug 7 19:24:20.664347220 (9h44m20.253692207s) [upload=10145]: Upload finished. err: <nil> Aug 7 19:24:57.194307899 (9h44m56.783652886s) [upload=10158]: Starting upload Aug 7 19:24:57.194352165 (9h44m56.783697142s) [upload=10158]: Done starting upload Aug 7 19:24:57.194396358 (9h44m56.783741346s) [upload=10158]: Uploading segments... Aug 7 19:24:57.194425755 (9h44m56.783770742s) [upload=10158]: Got next segment. Inline: true Aug 7 19:24:57.194429002 (9h44m56.783773989s) [upload=10158]: Waiting for error group managing segments... Aug 7 19:24:57.194459169 (9h44m56.783804156s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:57.194462526 (9h44m56.783807513s) [upload=10158]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:57.194464960 (9h44m56.783809947s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:57.508722153 (9h44m57.09806714s) [upload=10158]: Done uploading segments... Aug 7 19:24:57.508738245 (9h44m57.098083222s) [upload=10158]: Upload finished. err: <nil> Aug 7 19:25:23.090976846 (9h45m22.680321833s) [upload=10165]: Starting upload Aug 7 19:25:23.091016031 (9h45m22.680361018s) [upload=10165]: Done starting upload Aug 7 19:25:23.091078450 (9h45m22.680423427s) [upload=10165]: Uploading segments... Aug 7 19:25:23.091118557 (9h45m22.680463544s) [upload=10165]: Got next segment. Inline: true Aug 7 19:25:23.091122875 (9h45m22.680467852s) [upload=10165]: Waiting for error group managing segments... Aug 7 19:25:23.091155227 (9h45m22.680500214s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:25:23.091158914 (9h45m22.680503901s) [upload=10165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:25:23.091161098 (9h45m22.680506075s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:25:23.354144091 (9h45m22.943489078s) [upload=10165]: Done uploading segments... Aug 7 19:25:23.354161835 (9h45m22.943506822s) [upload=10165]: Upload finished. err: <nil> Aug 7 19:26:33.340151800 (9h46m32.929496787s) [upload=10181]: Starting upload Aug 7 19:26:33.340236143 (9h46m32.92958118s) [upload=10181]: Done starting upload Aug 7 19:26:33.340332376 (9h46m32.929677353s) [upload=10181]: Uploading segments... Aug 7 19:26:33.340373115 (9h46m32.929718112s) [upload=10181]: Got next segment. Inline: true Aug 7 19:26:33.340377673 (9h46m32.92972266s) [upload=10181]: Waiting for error group managing segments... Aug 7 19:26:33.340410516 (9h46m32.929755503s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:26:33.340414664 (9h46m32.929759651s) [upload=10181]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:26:33.340416918 (9h46m32.929761905s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:26:33.616120797 (9h46m33.205465824s) [upload=10181]: Done uploading segments... Aug 7 19:26:33.616142689 (9h46m33.205487666s) [upload=10181]: Upload finished. err: <nil> Aug 7 19:27:09.886832212 (9h47m9.476177199s) [upload=10193]: Starting upload Aug 7 19:27:09.886900242 (9h47m9.476245229s) [upload=10193]: Done starting upload Aug 7 19:27:09.886966249 (9h47m9.476311236s) [upload=10193]: Uploading segments... Aug 7 19:27:09.887030491 (9h47m9.476375478s) [upload=10193]: Got next segment. Inline: true Aug 7 19:27:09.887034268 (9h47m9.476379255s) [upload=10193]: Waiting for error group managing segments... Aug 7 19:27:09.887069426 (9h47m9.476414413s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:09.887098522 (9h47m9.476443509s) [upload=10193]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:09.887100906 (9h47m9.476445893s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Duration: 2.162339314s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X........X................................] duration: 5.939812377s FAIL (2025-08-07 19:29:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.939839478s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 1.900298631s FAIL (2025-08-07 19:29:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== b.BatchRequestItem_ObjectCommit Aug 7 19:19:25.172538463 (9h39m24.76188345s) [upload=10067]: Done uploading segments... Aug 7 19:19:25.172556097 (9h39m24.761901084s) [upload=10067]: Upload finished. err: <nil> Aug 7 19:19:59.108396078 (9h39m58.697741065s) [upload=10075]: Starting upload Aug 7 19:19:59.108438248 (9h39m58.697783225s) [upload=10075]: Done starting upload Aug 7 19:19:59.108486270 (9h39m58.697831257s) [upload=10075]: Uploading segments... Aug 7 19:19:59.108519834 (9h39m58.697864821s) [upload=10075]: Got next segment. Inline: true Aug 7 19:19:59.108523431 (9h39m58.697868418s) [upload=10075]: Waiting for error group managing segments... Aug 7 19:19:59.108555602 (9h39m58.697900589s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:19:59.108560452 (9h39m58.697905439s) [upload=10075]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:19:59.108563317 (9h39m58.697908305s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:19:59.535030122 (9h39m59.124375099s) [upload=10075]: Done uploading segments... Aug 7 19:19:59.535046735 (9h39m59.124391722s) [upload=10075]: Upload finished. err: <nil> Aug 7 19:21:07.972243470 (9h41m7.561588457s) [upload=10091]: Starting upload Aug 7 19:21:07.972303556 (9h41m7.561648543s) [upload=10091]: Done starting upload Aug 7 19:21:07.972352188 (9h41m7.561697175s) [upload=10091]: Uploading segments... Aug 7 19:21:07.972389430 (9h41m7.561734417s) [upload=10091]: Got next segment. Inline: true Aug 7 19:21:07.972393137 (9h41m7.561738124s) [upload=10091]: Waiting for error group managing segments... Aug 7 19:21:07.972440648 (9h41m7.561785635s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:21:07.972445257 (9h41m7.561790244s) [upload=10091]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:21:07.972447651 (9h41m7.561792639s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:21:08.256090200 (9h41m7.845435187s) [upload=10091]: Done uploading segments... Aug 7 19:21:08.256108215 (9h41m7.845453202s) [upload=10091]: Upload finished. err: <nil> Aug 7 19:22:04.622194388 (9h42m4.211539375s) [upload=10111]: Starting upload Aug 7 19:22:04.622243893 (9h42m4.21158888s) [upload=10111]: Done starting upload Aug 7 19:22:04.622314057 (9h42m4.211659044s) [upload=10111]: Uploading segments... Aug 7 19:22:04.622362129 (9h42m4.211707156s) [upload=10111]: Got next segment. Inline: true Aug 7 19:22:04.622368521 (9h42m4.211713528s) [upload=10111]: Waiting for error group managing segments... Aug 7 19:22:04.622413116 (9h42m4.211758104s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:22:04.622417394 (9h42m4.211762382s) [upload=10111]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:22:04.622419959 (9h42m4.211764946s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:22:04.930824935 (9h42m4.520169922s) [upload=10111]: Done uploading segments... Aug 7 19:22:04.930844082 (9h42m4.520189069s) [upload=10111]: Upload finished. err: <nil> Aug 7 19:23:18.125278170 (9h43m17.714623157s) [upload=10129]: Starting upload Aug 7 19:23:18.125322425 (9h43m17.714667412s) [upload=10129]: Done starting upload Aug 7 19:23:18.125398790 (9h43m17.714743768s) [upload=10129]: Uploading segments... Aug 7 19:23:18.125427335 (9h43m17.714772322s) [upload=10129]: Got next segment. Inline: true Aug 7 19:23:18.125430982 (9h43m17.714775969s) [upload=10129]: Waiting for error group managing segments... Aug 7 19:23:18.125486578 (9h43m17.714831565s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:23:18.125491038 (9h43m17.714836025s) [upload=10129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:23:18.125493492 (9h43m17.714838479s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:23:18.413472808 (9h43m18.002817795s) [upload=10129]: Done uploading segments... Aug 7 19:23:18.413488809 (9h43m18.002833786s) [upload=10129]: Upload finished. err: <nil> Aug 7 19:24:20.169825506 (9h44m19.759170483s) [upload=10145]: Starting upload Aug 7 19:24:20.169871795 (9h44m19.759216772s) [upload=10145]: Done starting upload Aug 7 19:24:20.169923194 (9h44m19.759268171s) [upload=10145]: Uploading segments... Aug 7 19:24:20.169954353 (9h44m19.75929934s) [upload=10145]: Got next segment. Inline: true Aug 7 19:24:20.169958020 (9h44m19.759303007s) [upload=10145]: Waiting for error group managing segments... Aug 7 19:24:20.169989220 (9h44m19.759334197s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:20.169993168 (9h44m19.759338155s) [upload=10145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:20.169995552 (9h44m19.759340539s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:20.664326400 (9h44m20.253671387s) [upload=10145]: Done uploading segments... Aug 7 19:24:20.664347220 (9h44m20.253692207s) [upload=10145]: Upload finished. err: <nil> Aug 7 19:24:57.194307899 (9h44m56.783652886s) [upload=10158]: Starting upload Aug 7 19:24:57.194352165 (9h44m56.783697142s) [upload=10158]: Done starting upload Aug 7 19:24:57.194396358 (9h44m56.783741346s) [upload=10158]: Uploading segments... Aug 7 19:24:57.194425755 (9h44m56.783770742s) [upload=10158]: Got next segment. Inline: true Aug 7 19:24:57.194429002 (9h44m56.783773989s) [upload=10158]: Waiting for error group managing segments... Aug 7 19:24:57.194459169 (9h44m56.783804156s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:57.194462526 (9h44m56.783807513s) [upload=10158]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:57.194464960 (9h44m56.783809947s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:57.508722153 (9h44m57.09806714s) [upload=10158]: Done uploading segments... Aug 7 19:24:57.508738245 (9h44m57.098083222s) [upload=10158]: Upload finished. err: <nil> Aug 7 19:25:23.090976846 (9h45m22.680321833s) [upload=10165]: Starting upload Aug 7 19:25:23.091016031 (9h45m22.680361018s) [upload=10165]: Done starting upload Aug 7 19:25:23.091078450 (9h45m22.680423427s) [upload=10165]: Uploading segments... Aug 7 19:25:23.091118557 (9h45m22.680463544s) [upload=10165]: Got next segment. Inline: true Aug 7 19:25:23.091122875 (9h45m22.680467852s) [upload=10165]: Waiting for error group managing segments... Aug 7 19:25:23.091155227 (9h45m22.680500214s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:25:23.091158914 (9h45m22.680503901s) [upload=10165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:25:23.091161098 (9h45m22.680506075s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:25:23.354144091 (9h45m22.943489078s) [upload=10165]: Done uploading segments... Aug 7 19:25:23.354161835 (9h45m22.943506822s) [upload=10165]: Upload finished. err: <nil> Aug 7 19:26:33.340151800 (9h46m32.929496787s) [upload=10181]: Starting upload Aug 7 19:26:33.340236143 (9h46m32.92958118s) [upload=10181]: Done starting upload Aug 7 19:26:33.340332376 (9h46m32.929677353s) [upload=10181]: Uploading segments... Aug 7 19:26:33.340373115 (9h46m32.929718112s) [upload=10181]: Got next segment. Inline: true Aug 7 19:26:33.340377673 (9h46m32.92972266s) [upload=10181]: Waiting for error group managing segments... Aug 7 19:26:33.340410516 (9h46m32.929755503s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:26:33.340414664 (9h46m32.929759651s) [upload=10181]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:26:33.340416918 (9h46m32.929761905s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:26:33.616120797 (9h46m33.205465824s) [upload=10181]: Done uploading segments... Aug 7 19:26:33.616142689 (9h46m33.205487666s) [upload=10181]: Upload finished. err: <nil> Aug 7 19:27:09.886832212 (9h47m9.476177199s) [upload=10193]: Starting upload Aug 7 19:27:09.886900242 (9h47m9.476245229s) [upload=10193]: Done starting upload Aug 7 19:27:09.886966249 (9h47m9.476311236s) [upload=10193]: Uploading segments... Aug 7 19:27:09.887030491 (9h47m9.476375478s) [upload=10193]: Got next segment. Inline: true Aug 7 19:27:09.887034268 (9h47m9.476379255s) [upload=10193]: Waiting for error group managing segments... Aug 7 19:27:09.887069426 (9h47m9.476414413s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:09.887098522 (9h47m9.476443509s) [upload=10193]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:09.887100906 (9h47m9.476445893s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Duration: 1.900320383s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.94131062s FAIL (2025-08-07 19:29:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== b.BatchRequestItem_ObjectCommit Aug 7 19:19:25.172538463 (9h39m24.76188345s) [upload=10067]: Done uploading segments... Aug 7 19:19:25.172556097 (9h39m24.761901084s) [upload=10067]: Upload finished. err: <nil> Aug 7 19:19:59.108396078 (9h39m58.697741065s) [upload=10075]: Starting upload Aug 7 19:19:59.108438248 (9h39m58.697783225s) [upload=10075]: Done starting upload Aug 7 19:19:59.108486270 (9h39m58.697831257s) [upload=10075]: Uploading segments... Aug 7 19:19:59.108519834 (9h39m58.697864821s) [upload=10075]: Got next segment. Inline: true Aug 7 19:19:59.108523431 (9h39m58.697868418s) [upload=10075]: Waiting for error group managing segments... Aug 7 19:19:59.108555602 (9h39m58.697900589s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:19:59.108560452 (9h39m58.697905439s) [upload=10075]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:19:59.108563317 (9h39m58.697908305s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:19:59.535030122 (9h39m59.124375099s) [upload=10075]: Done uploading segments... Aug 7 19:19:59.535046735 (9h39m59.124391722s) [upload=10075]: Upload finished. err: <nil> Aug 7 19:21:07.972243470 (9h41m7.561588457s) [upload=10091]: Starting upload Aug 7 19:21:07.972303556 (9h41m7.561648543s) [upload=10091]: Done starting upload Aug 7 19:21:07.972352188 (9h41m7.561697175s) [upload=10091]: Uploading segments... Aug 7 19:21:07.972389430 (9h41m7.561734417s) [upload=10091]: Got next segment. Inline: true Aug 7 19:21:07.972393137 (9h41m7.561738124s) [upload=10091]: Waiting for error group managing segments... Aug 7 19:21:07.972440648 (9h41m7.561785635s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:21:07.972445257 (9h41m7.561790244s) [upload=10091]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:21:07.972447651 (9h41m7.561792639s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:21:08.256090200 (9h41m7.845435187s) [upload=10091]: Done uploading segments... Aug 7 19:21:08.256108215 (9h41m7.845453202s) [upload=10091]: Upload finished. err: <nil> Aug 7 19:22:04.622194388 (9h42m4.211539375s) [upload=10111]: Starting upload Aug 7 19:22:04.622243893 (9h42m4.21158888s) [upload=10111]: Done starting upload Aug 7 19:22:04.622314057 (9h42m4.211659044s) [upload=10111]: Uploading segments... Aug 7 19:22:04.622362129 (9h42m4.211707156s) [upload=10111]: Got next segment. Inline: true Aug 7 19:22:04.622368521 (9h42m4.211713528s) [upload=10111]: Waiting for error group managing segments... Aug 7 19:22:04.622413116 (9h42m4.211758104s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:22:04.622417394 (9h42m4.211762382s) [upload=10111]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:22:04.622419959 (9h42m4.211764946s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:22:04.930824935 (9h42m4.520169922s) [upload=10111]: Done uploading segments... Aug 7 19:22:04.930844082 (9h42m4.520189069s) [upload=10111]: Upload finished. err: <nil> Aug 7 19:23:18.125278170 (9h43m17.714623157s) [upload=10129]: Starting upload Aug 7 19:23:18.125322425 (9h43m17.714667412s) [upload=10129]: Done starting upload Aug 7 19:23:18.125398790 (9h43m17.714743768s) [upload=10129]: Uploading segments... Aug 7 19:23:18.125427335 (9h43m17.714772322s) [upload=10129]: Got next segment. Inline: true Aug 7 19:23:18.125430982 (9h43m17.714775969s) [upload=10129]: Waiting for error group managing segments... Aug 7 19:23:18.125486578 (9h43m17.714831565s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:23:18.125491038 (9h43m17.714836025s) [upload=10129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:23:18.125493492 (9h43m17.714838479s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:23:18.413472808 (9h43m18.002817795s) [upload=10129]: Done uploading segments... Aug 7 19:23:18.413488809 (9h43m18.002833786s) [upload=10129]: Upload finished. err: <nil> Aug 7 19:24:20.169825506 (9h44m19.759170483s) [upload=10145]: Starting upload Aug 7 19:24:20.169871795 (9h44m19.759216772s) [upload=10145]: Done starting upload Aug 7 19:24:20.169923194 (9h44m19.759268171s) [upload=10145]: Uploading segments... Aug 7 19:24:20.169954353 (9h44m19.75929934s) [upload=10145]: Got next segment. Inline: true Aug 7 19:24:20.169958020 (9h44m19.759303007s) [upload=10145]: Waiting for error group managing segments... Aug 7 19:24:20.169989220 (9h44m19.759334197s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:20.169993168 (9h44m19.759338155s) [upload=10145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:20.169995552 (9h44m19.759340539s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:20.664326400 (9h44m20.253671387s) [upload=10145]: Done uploading segments... Aug 7 19:24:20.664347220 (9h44m20.253692207s) [upload=10145]: Upload finished. err: <nil> Aug 7 19:24:57.194307899 (9h44m56.783652886s) [upload=10158]: Starting upload Aug 7 19:24:57.194352165 (9h44m56.783697142s) [upload=10158]: Done starting upload Aug 7 19:24:57.194396358 (9h44m56.783741346s) [upload=10158]: Uploading segments... Aug 7 19:24:57.194425755 (9h44m56.783770742s) [upload=10158]: Got next segment. Inline: true Aug 7 19:24:57.194429002 (9h44m56.783773989s) [upload=10158]: Waiting for error group managing segments... Aug 7 19:24:57.194459169 (9h44m56.783804156s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:57.194462526 (9h44m56.783807513s) [upload=10158]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:57.194464960 (9h44m56.783809947s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:57.508722153 (9h44m57.09806714s) [upload=10158]: Done uploading segments... Aug 7 19:24:57.508738245 (9h44m57.098083222s) [upload=10158]: Upload finished. err: <nil> Aug 7 19:25:23.090976846 (9h45m22.680321833s) [upload=10165]: Starting upload Aug 7 19:25:23.091016031 (9h45m22.680361018s) [upload=10165]: Done starting upload Aug 7 19:25:23.091078450 (9h45m22.680423427s) [upload=10165]: Uploading segments... Aug 7 19:25:23.091118557 (9h45m22.680463544s) [upload=10165]: Got next segment. Inline: true Aug 7 19:25:23.091122875 (9h45m22.680467852s) [upload=10165]: Waiting for error group managing segments... Aug 7 19:25:23.091155227 (9h45m22.680500214s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:25:23.091158914 (9h45m22.680503901s) [upload=10165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:25:23.091161098 (9h45m22.680506075s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:25:23.354144091 (9h45m22.943489078s) [upload=10165]: Done uploading segments... Aug 7 19:25:23.354161835 (9h45m22.943506822s) [upload=10165]: Upload finished. err: <nil> Aug 7 19:26:33.340151800 (9h46m32.929496787s) [upload=10181]: Starting upload Aug 7 19:26:33.340236143 (9h46m32.92958118s) [upload=10181]: Done starting upload Aug 7 19:26:33.340332376 (9h46m32.929677353s) [upload=10181]: Uploading segments... Aug 7 19:26:33.340373115 (9h46m32.929718112s) [upload=10181]: Got next segment. Inline: true Aug 7 19:26:33.340377673 (9h46m32.92972266s) [upload=10181]: Waiting for error group managing segments... Aug 7 19:26:33.340410516 (9h46m32.929755503s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:26:33.340414664 (9h46m32.929759651s) [upload=10181]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:26:33.340416918 (9h46m32.929761905s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:26:33.616120797 (9h46m33.205465824s) [upload=10181]: Done uploading segments... Aug 7 19:26:33.616142689 (9h46m33.205487666s) [upload=10181]: Upload finished. err: <nil> Aug 7 19:27:09.886832212 (9h47m9.476177199s) [upload=10193]: Starting upload Aug 7 19:27:09.886900242 (9h47m9.476245229s) [upload=10193]: Done starting upload Aug 7 19:27:09.886966249 (9h47m9.476311236s) [upload=10193]: Uploading segments... Aug 7 19:27:09.887030491 (9h47m9.476375478s) [upload=10193]: Got next segment. Inline: true Aug 7 19:27:09.887034268 (9h47m9.476379255s) [upload=10193]: Waiting for error group managing segments... Aug 7 19:27:09.887069426 (9h47m9.476414413s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:09.887098522 (9h47m9.476443509s) [upload=10193]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:09.887100906 (9h47m9.476445893s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Duration: 1.941337974s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 1.86522625s FAIL (2025-08-07 19:28:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== b.BatchRequestItem_ObjectCommit Aug 7 19:19:25.172538463 (9h39m24.76188345s) [upload=10067]: Done uploading segments... Aug 7 19:19:25.172556097 (9h39m24.761901084s) [upload=10067]: Upload finished. err: <nil> Aug 7 19:19:59.108396078 (9h39m58.697741065s) [upload=10075]: Starting upload Aug 7 19:19:59.108438248 (9h39m58.697783225s) [upload=10075]: Done starting upload Aug 7 19:19:59.108486270 (9h39m58.697831257s) [upload=10075]: Uploading segments... Aug 7 19:19:59.108519834 (9h39m58.697864821s) [upload=10075]: Got next segment. Inline: true Aug 7 19:19:59.108523431 (9h39m58.697868418s) [upload=10075]: Waiting for error group managing segments... Aug 7 19:19:59.108555602 (9h39m58.697900589s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:19:59.108560452 (9h39m58.697905439s) [upload=10075]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:19:59.108563317 (9h39m58.697908305s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:19:59.535030122 (9h39m59.124375099s) [upload=10075]: Done uploading segments... Aug 7 19:19:59.535046735 (9h39m59.124391722s) [upload=10075]: Upload finished. err: <nil> Aug 7 19:21:07.972243470 (9h41m7.561588457s) [upload=10091]: Starting upload Aug 7 19:21:07.972303556 (9h41m7.561648543s) [upload=10091]: Done starting upload Aug 7 19:21:07.972352188 (9h41m7.561697175s) [upload=10091]: Uploading segments... Aug 7 19:21:07.972389430 (9h41m7.561734417s) [upload=10091]: Got next segment. Inline: true Aug 7 19:21:07.972393137 (9h41m7.561738124s) [upload=10091]: Waiting for error group managing segments... Aug 7 19:21:07.972440648 (9h41m7.561785635s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:21:07.972445257 (9h41m7.561790244s) [upload=10091]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:21:07.972447651 (9h41m7.561792639s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:21:08.256090200 (9h41m7.845435187s) [upload=10091]: Done uploading segments... Aug 7 19:21:08.256108215 (9h41m7.845453202s) [upload=10091]: Upload finished. err: <nil> Aug 7 19:22:04.622194388 (9h42m4.211539375s) [upload=10111]: Starting upload Aug 7 19:22:04.622243893 (9h42m4.21158888s) [upload=10111]: Done starting upload Aug 7 19:22:04.622314057 (9h42m4.211659044s) [upload=10111]: Uploading segments... Aug 7 19:22:04.622362129 (9h42m4.211707156s) [upload=10111]: Got next segment. Inline: true Aug 7 19:22:04.622368521 (9h42m4.211713528s) [upload=10111]: Waiting for error group managing segments... Aug 7 19:22:04.622413116 (9h42m4.211758104s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:22:04.622417394 (9h42m4.211762382s) [upload=10111]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:22:04.622419959 (9h42m4.211764946s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:22:04.930824935 (9h42m4.520169922s) [upload=10111]: Done uploading segments... Aug 7 19:22:04.930844082 (9h42m4.520189069s) [upload=10111]: Upload finished. err: <nil> Aug 7 19:23:18.125278170 (9h43m17.714623157s) [upload=10129]: Starting upload Aug 7 19:23:18.125322425 (9h43m17.714667412s) [upload=10129]: Done starting upload Aug 7 19:23:18.125398790 (9h43m17.714743768s) [upload=10129]: Uploading segments... Aug 7 19:23:18.125427335 (9h43m17.714772322s) [upload=10129]: Got next segment. Inline: true Aug 7 19:23:18.125430982 (9h43m17.714775969s) [upload=10129]: Waiting for error group managing segments... Aug 7 19:23:18.125486578 (9h43m17.714831565s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:23:18.125491038 (9h43m17.714836025s) [upload=10129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:23:18.125493492 (9h43m17.714838479s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:23:18.413472808 (9h43m18.002817795s) [upload=10129]: Done uploading segments... Aug 7 19:23:18.413488809 (9h43m18.002833786s) [upload=10129]: Upload finished. err: <nil> Aug 7 19:24:20.169825506 (9h44m19.759170483s) [upload=10145]: Starting upload Aug 7 19:24:20.169871795 (9h44m19.759216772s) [upload=10145]: Done starting upload Aug 7 19:24:20.169923194 (9h44m19.759268171s) [upload=10145]: Uploading segments... Aug 7 19:24:20.169954353 (9h44m19.75929934s) [upload=10145]: Got next segment. Inline: true Aug 7 19:24:20.169958020 (9h44m19.759303007s) [upload=10145]: Waiting for error group managing segments... Aug 7 19:24:20.169989220 (9h44m19.759334197s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:20.169993168 (9h44m19.759338155s) [upload=10145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:20.169995552 (9h44m19.759340539s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:20.664326400 (9h44m20.253671387s) [upload=10145]: Done uploading segments... Aug 7 19:24:20.664347220 (9h44m20.253692207s) [upload=10145]: Upload finished. err: <nil> Aug 7 19:24:57.194307899 (9h44m56.783652886s) [upload=10158]: Starting upload Aug 7 19:24:57.194352165 (9h44m56.783697142s) [upload=10158]: Done starting upload Aug 7 19:24:57.194396358 (9h44m56.783741346s) [upload=10158]: Uploading segments... Aug 7 19:24:57.194425755 (9h44m56.783770742s) [upload=10158]: Got next segment. Inline: true Aug 7 19:24:57.194429002 (9h44m56.783773989s) [upload=10158]: Waiting for error group managing segments... Aug 7 19:24:57.194459169 (9h44m56.783804156s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:57.194462526 (9h44m56.783807513s) [upload=10158]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:57.194464960 (9h44m56.783809947s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:57.508722153 (9h44m57.09806714s) [upload=10158]: Done uploading segments... Aug 7 19:24:57.508738245 (9h44m57.098083222s) [upload=10158]: Upload finished. err: <nil> Aug 7 19:25:23.090976846 (9h45m22.680321833s) [upload=10165]: Starting upload Aug 7 19:25:23.091016031 (9h45m22.680361018s) [upload=10165]: Done starting upload Aug 7 19:25:23.091078450 (9h45m22.680423427s) [upload=10165]: Uploading segments... Aug 7 19:25:23.091118557 (9h45m22.680463544s) [upload=10165]: Got next segment. Inline: true Aug 7 19:25:23.091122875 (9h45m22.680467852s) [upload=10165]: Waiting for error group managing segments... Aug 7 19:25:23.091155227 (9h45m22.680500214s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:25:23.091158914 (9h45m22.680503901s) [upload=10165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:25:23.091161098 (9h45m22.680506075s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:25:23.354144091 (9h45m22.943489078s) [upload=10165]: Done uploading segments... Aug 7 19:25:23.354161835 (9h45m22.943506822s) [upload=10165]: Upload finished. err: <nil> Aug 7 19:26:33.340151800 (9h46m32.929496787s) [upload=10181]: Starting upload Aug 7 19:26:33.340236143 (9h46m32.92958118s) [upload=10181]: Done starting upload Aug 7 19:26:33.340332376 (9h46m32.929677353s) [upload=10181]: Uploading segments... Aug 7 19:26:33.340373115 (9h46m32.929718112s) [upload=10181]: Got next segment. Inline: true Aug 7 19:26:33.340377673 (9h46m32.92972266s) [upload=10181]: Waiting for error group managing segments... Aug 7 19:26:33.340410516 (9h46m32.929755503s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:26:33.340414664 (9h46m32.929759651s) [upload=10181]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:26:33.340416918 (9h46m32.929761905s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:26:33.616120797 (9h46m33.205465824s) [upload=10181]: Done uploading segments... Aug 7 19:26:33.616142689 (9h46m33.205487666s) [upload=10181]: Upload finished. err: <nil> Aug 7 19:27:09.886832212 (9h47m9.476177199s) [upload=10193]: Starting upload Aug 7 19:27:09.886900242 (9h47m9.476245229s) [upload=10193]: Done starting upload Aug 7 19:27:09.886966249 (9h47m9.476311236s) [upload=10193]: Uploading segments... Aug 7 19:27:09.887030491 (9h47m9.476375478s) [upload=10193]: Got next segment. Inline: true Aug 7 19:27:09.887034268 (9h47m9.476379255s) [upload=10193]: Waiting for error group managing segments... Aug 7 19:27:09.887069426 (9h47m9.476414413s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:09.887098522 (9h47m9.476443509s) [upload=10193]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:09.887100906 (9h47m9.476445893s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Duration: 1.865282777s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................X........X...............................] duration: 6.22398703s FAIL (2025-08-07 19:28:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 6.224013941s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X.........................................................X.] duration: 2.158642727s FAIL (2025-08-07 19:28:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== b.BatchRequestItem_ObjectCommit Aug 7 19:19:25.172538463 (9h39m24.76188345s) [upload=10067]: Done uploading segments... Aug 7 19:19:25.172556097 (9h39m24.761901084s) [upload=10067]: Upload finished. err: <nil> Aug 7 19:19:59.108396078 (9h39m58.697741065s) [upload=10075]: Starting upload Aug 7 19:19:59.108438248 (9h39m58.697783225s) [upload=10075]: Done starting upload Aug 7 19:19:59.108486270 (9h39m58.697831257s) [upload=10075]: Uploading segments... Aug 7 19:19:59.108519834 (9h39m58.697864821s) [upload=10075]: Got next segment. Inline: true Aug 7 19:19:59.108523431 (9h39m58.697868418s) [upload=10075]: Waiting for error group managing segments... Aug 7 19:19:59.108555602 (9h39m58.697900589s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:19:59.108560452 (9h39m58.697905439s) [upload=10075]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:19:59.108563317 (9h39m58.697908305s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:19:59.535030122 (9h39m59.124375099s) [upload=10075]: Done uploading segments... Aug 7 19:19:59.535046735 (9h39m59.124391722s) [upload=10075]: Upload finished. err: <nil> Aug 7 19:21:07.972243470 (9h41m7.561588457s) [upload=10091]: Starting upload Aug 7 19:21:07.972303556 (9h41m7.561648543s) [upload=10091]: Done starting upload Aug 7 19:21:07.972352188 (9h41m7.561697175s) [upload=10091]: Uploading segments... Aug 7 19:21:07.972389430 (9h41m7.561734417s) [upload=10091]: Got next segment. Inline: true Aug 7 19:21:07.972393137 (9h41m7.561738124s) [upload=10091]: Waiting for error group managing segments... Aug 7 19:21:07.972440648 (9h41m7.561785635s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:21:07.972445257 (9h41m7.561790244s) [upload=10091]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:21:07.972447651 (9h41m7.561792639s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:21:08.256090200 (9h41m7.845435187s) [upload=10091]: Done uploading segments... Aug 7 19:21:08.256108215 (9h41m7.845453202s) [upload=10091]: Upload finished. err: <nil> Aug 7 19:22:04.622194388 (9h42m4.211539375s) [upload=10111]: Starting upload Aug 7 19:22:04.622243893 (9h42m4.21158888s) [upload=10111]: Done starting upload Aug 7 19:22:04.622314057 (9h42m4.211659044s) [upload=10111]: Uploading segments... Aug 7 19:22:04.622362129 (9h42m4.211707156s) [upload=10111]: Got next segment. Inline: true Aug 7 19:22:04.622368521 (9h42m4.211713528s) [upload=10111]: Waiting for error group managing segments... Aug 7 19:22:04.622413116 (9h42m4.211758104s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:22:04.622417394 (9h42m4.211762382s) [upload=10111]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:22:04.622419959 (9h42m4.211764946s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:22:04.930824935 (9h42m4.520169922s) [upload=10111]: Done uploading segments... Aug 7 19:22:04.930844082 (9h42m4.520189069s) [upload=10111]: Upload finished. err: <nil> Aug 7 19:23:18.125278170 (9h43m17.714623157s) [upload=10129]: Starting upload Aug 7 19:23:18.125322425 (9h43m17.714667412s) [upload=10129]: Done starting upload Aug 7 19:23:18.125398790 (9h43m17.714743768s) [upload=10129]: Uploading segments... Aug 7 19:23:18.125427335 (9h43m17.714772322s) [upload=10129]: Got next segment. Inline: true Aug 7 19:23:18.125430982 (9h43m17.714775969s) [upload=10129]: Waiting for error group managing segments... Aug 7 19:23:18.125486578 (9h43m17.714831565s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:23:18.125491038 (9h43m17.714836025s) [upload=10129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:23:18.125493492 (9h43m17.714838479s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:23:18.413472808 (9h43m18.002817795s) [upload=10129]: Done uploading segments... Aug 7 19:23:18.413488809 (9h43m18.002833786s) [upload=10129]: Upload finished. err: <nil> Aug 7 19:24:20.169825506 (9h44m19.759170483s) [upload=10145]: Starting upload Aug 7 19:24:20.169871795 (9h44m19.759216772s) [upload=10145]: Done starting upload Aug 7 19:24:20.169923194 (9h44m19.759268171s) [upload=10145]: Uploading segments... Aug 7 19:24:20.169954353 (9h44m19.75929934s) [upload=10145]: Got next segment. Inline: true Aug 7 19:24:20.169958020 (9h44m19.759303007s) [upload=10145]: Waiting for error group managing segments... Aug 7 19:24:20.169989220 (9h44m19.759334197s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:20.169993168 (9h44m19.759338155s) [upload=10145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:20.169995552 (9h44m19.759340539s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:20.664326400 (9h44m20.253671387s) [upload=10145]: Done uploading segments... Aug 7 19:24:20.664347220 (9h44m20.253692207s) [upload=10145]: Upload finished. err: <nil> Aug 7 19:24:57.194307899 (9h44m56.783652886s) [upload=10158]: Starting upload Aug 7 19:24:57.194352165 (9h44m56.783697142s) [upload=10158]: Done starting upload Aug 7 19:24:57.194396358 (9h44m56.783741346s) [upload=10158]: Uploading segments... Aug 7 19:24:57.194425755 (9h44m56.783770742s) [upload=10158]: Got next segment. Inline: true Aug 7 19:24:57.194429002 (9h44m56.783773989s) [upload=10158]: Waiting for error group managing segments... Aug 7 19:24:57.194459169 (9h44m56.783804156s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:57.194462526 (9h44m56.783807513s) [upload=10158]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:57.194464960 (9h44m56.783809947s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:57.508722153 (9h44m57.09806714s) [upload=10158]: Done uploading segments... Aug 7 19:24:57.508738245 (9h44m57.098083222s) [upload=10158]: Upload finished. err: <nil> Aug 7 19:25:23.090976846 (9h45m22.680321833s) [upload=10165]: Starting upload Aug 7 19:25:23.091016031 (9h45m22.680361018s) [upload=10165]: Done starting upload Aug 7 19:25:23.091078450 (9h45m22.680423427s) [upload=10165]: Uploading segments... Aug 7 19:25:23.091118557 (9h45m22.680463544s) [upload=10165]: Got next segment. Inline: true Aug 7 19:25:23.091122875 (9h45m22.680467852s) [upload=10165]: Waiting for error group managing segments... Aug 7 19:25:23.091155227 (9h45m22.680500214s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:25:23.091158914 (9h45m22.680503901s) [upload=10165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:25:23.091161098 (9h45m22.680506075s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:25:23.354144091 (9h45m22.943489078s) [upload=10165]: Done uploading segments... Aug 7 19:25:23.354161835 (9h45m22.943506822s) [upload=10165]: Upload finished. err: <nil> Aug 7 19:26:33.340151800 (9h46m32.929496787s) [upload=10181]: Starting upload Aug 7 19:26:33.340236143 (9h46m32.92958118s) [upload=10181]: Done starting upload Aug 7 19:26:33.340332376 (9h46m32.929677353s) [upload=10181]: Uploading segments... Aug 7 19:26:33.340373115 (9h46m32.929718112s) [upload=10181]: Got next segment. Inline: true Aug 7 19:26:33.340377673 (9h46m32.92972266s) [upload=10181]: Waiting for error group managing segments... Aug 7 19:26:33.340410516 (9h46m32.929755503s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:26:33.340414664 (9h46m32.929759651s) [upload=10181]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:26:33.340416918 (9h46m32.929761905s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:26:33.616120797 (9h46m33.205465824s) [upload=10181]: Done uploading segments... Aug 7 19:26:33.616142689 (9h46m33.205487666s) [upload=10181]: Upload finished. err: <nil> Aug 7 19:27:09.886832212 (9h47m9.476177199s) [upload=10193]: Starting upload Aug 7 19:27:09.886900242 (9h47m9.476245229s) [upload=10193]: Done starting upload Aug 7 19:27:09.886966249 (9h47m9.476311236s) [upload=10193]: Uploading segments... Aug 7 19:27:09.887030491 (9h47m9.476375478s) [upload=10193]: Got next segment. Inline: true Aug 7 19:27:09.887034268 (9h47m9.476379255s) [upload=10193]: Waiting for error group managing segments... Aug 7 19:27:09.887069426 (9h47m9.476414413s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:09.887098522 (9h47m9.476443509s) [upload=10193]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:09.887100906 (9h47m9.476445893s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Duration: 2.158665371s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................X........X..............................] duration: 5.95859339s FAIL (2025-08-07 19:27:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.958620632s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X.........................................................X] duration: 868.434432ms FAIL (2025-08-07 19:27:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== b.BatchRequestItem_ObjectCommit Aug 7 19:19:25.172538463 (9h39m24.76188345s) [upload=10067]: Done uploading segments... Aug 7 19:19:25.172556097 (9h39m24.761901084s) [upload=10067]: Upload finished. err: <nil> Aug 7 19:19:59.108396078 (9h39m58.697741065s) [upload=10075]: Starting upload Aug 7 19:19:59.108438248 (9h39m58.697783225s) [upload=10075]: Done starting upload Aug 7 19:19:59.108486270 (9h39m58.697831257s) [upload=10075]: Uploading segments... Aug 7 19:19:59.108519834 (9h39m58.697864821s) [upload=10075]: Got next segment. Inline: true Aug 7 19:19:59.108523431 (9h39m58.697868418s) [upload=10075]: Waiting for error group managing segments... Aug 7 19:19:59.108555602 (9h39m58.697900589s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:19:59.108560452 (9h39m58.697905439s) [upload=10075]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:19:59.108563317 (9h39m58.697908305s) [upload=10075]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:19:59.535030122 (9h39m59.124375099s) [upload=10075]: Done uploading segments... Aug 7 19:19:59.535046735 (9h39m59.124391722s) [upload=10075]: Upload finished. err: <nil> Aug 7 19:21:07.972243470 (9h41m7.561588457s) [upload=10091]: Starting upload Aug 7 19:21:07.972303556 (9h41m7.561648543s) [upload=10091]: Done starting upload Aug 7 19:21:07.972352188 (9h41m7.561697175s) [upload=10091]: Uploading segments... Aug 7 19:21:07.972389430 (9h41m7.561734417s) [upload=10091]: Got next segment. Inline: true Aug 7 19:21:07.972393137 (9h41m7.561738124s) [upload=10091]: Waiting for error group managing segments... Aug 7 19:21:07.972440648 (9h41m7.561785635s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:21:07.972445257 (9h41m7.561790244s) [upload=10091]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:21:07.972447651 (9h41m7.561792639s) [upload=10091]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:21:08.256090200 (9h41m7.845435187s) [upload=10091]: Done uploading segments... Aug 7 19:21:08.256108215 (9h41m7.845453202s) [upload=10091]: Upload finished. err: <nil> Aug 7 19:22:04.622194388 (9h42m4.211539375s) [upload=10111]: Starting upload Aug 7 19:22:04.622243893 (9h42m4.21158888s) [upload=10111]: Done starting upload Aug 7 19:22:04.622314057 (9h42m4.211659044s) [upload=10111]: Uploading segments... Aug 7 19:22:04.622362129 (9h42m4.211707156s) [upload=10111]: Got next segment. Inline: true Aug 7 19:22:04.622368521 (9h42m4.211713528s) [upload=10111]: Waiting for error group managing segments... Aug 7 19:22:04.622413116 (9h42m4.211758104s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:22:04.622417394 (9h42m4.211762382s) [upload=10111]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:22:04.622419959 (9h42m4.211764946s) [upload=10111]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:22:04.930824935 (9h42m4.520169922s) [upload=10111]: Done uploading segments... Aug 7 19:22:04.930844082 (9h42m4.520189069s) [upload=10111]: Upload finished. err: <nil> Aug 7 19:23:18.125278170 (9h43m17.714623157s) [upload=10129]: Starting upload Aug 7 19:23:18.125322425 (9h43m17.714667412s) [upload=10129]: Done starting upload Aug 7 19:23:18.125398790 (9h43m17.714743768s) [upload=10129]: Uploading segments... Aug 7 19:23:18.125427335 (9h43m17.714772322s) [upload=10129]: Got next segment. Inline: true Aug 7 19:23:18.125430982 (9h43m17.714775969s) [upload=10129]: Waiting for error group managing segments... Aug 7 19:23:18.125486578 (9h43m17.714831565s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:23:18.125491038 (9h43m17.714836025s) [upload=10129]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:23:18.125493492 (9h43m17.714838479s) [upload=10129]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:23:18.413472808 (9h43m18.002817795s) [upload=10129]: Done uploading segments... Aug 7 19:23:18.413488809 (9h43m18.002833786s) [upload=10129]: Upload finished. err: <nil> Aug 7 19:24:20.169825506 (9h44m19.759170483s) [upload=10145]: Starting upload Aug 7 19:24:20.169871795 (9h44m19.759216772s) [upload=10145]: Done starting upload Aug 7 19:24:20.169923194 (9h44m19.759268171s) [upload=10145]: Uploading segments... Aug 7 19:24:20.169954353 (9h44m19.75929934s) [upload=10145]: Got next segment. Inline: true Aug 7 19:24:20.169958020 (9h44m19.759303007s) [upload=10145]: Waiting for error group managing segments... Aug 7 19:24:20.169989220 (9h44m19.759334197s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:20.169993168 (9h44m19.759338155s) [upload=10145]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:20.169995552 (9h44m19.759340539s) [upload=10145]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:20.664326400 (9h44m20.253671387s) [upload=10145]: Done uploading segments... Aug 7 19:24:20.664347220 (9h44m20.253692207s) [upload=10145]: Upload finished. err: <nil> Aug 7 19:24:57.194307899 (9h44m56.783652886s) [upload=10158]: Starting upload Aug 7 19:24:57.194352165 (9h44m56.783697142s) [upload=10158]: Done starting upload Aug 7 19:24:57.194396358 (9h44m56.783741346s) [upload=10158]: Uploading segments... Aug 7 19:24:57.194425755 (9h44m56.783770742s) [upload=10158]: Got next segment. Inline: true Aug 7 19:24:57.194429002 (9h44m56.783773989s) [upload=10158]: Waiting for error group managing segments... Aug 7 19:24:57.194459169 (9h44m56.783804156s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:24:57.194462526 (9h44m56.783807513s) [upload=10158]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:24:57.194464960 (9h44m56.783809947s) [upload=10158]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:24:57.508722153 (9h44m57.09806714s) [upload=10158]: Done uploading segments... Aug 7 19:24:57.508738245 (9h44m57.098083222s) [upload=10158]: Upload finished. err: <nil> Aug 7 19:25:23.090976846 (9h45m22.680321833s) [upload=10165]: Starting upload Aug 7 19:25:23.091016031 (9h45m22.680361018s) [upload=10165]: Done starting upload Aug 7 19:25:23.091078450 (9h45m22.680423427s) [upload=10165]: Uploading segments... Aug 7 19:25:23.091118557 (9h45m22.680463544s) [upload=10165]: Got next segment. Inline: true Aug 7 19:25:23.091122875 (9h45m22.680467852s) [upload=10165]: Waiting for error group managing segments... Aug 7 19:25:23.091155227 (9h45m22.680500214s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:25:23.091158914 (9h45m22.680503901s) [upload=10165]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:25:23.091161098 (9h45m22.680506075s) [upload=10165]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:25:23.354144091 (9h45m22.943489078s) [upload=10165]: Done uploading segments... Aug 7 19:25:23.354161835 (9h45m22.943506822s) [upload=10165]: Upload finished. err: <nil> Aug 7 19:26:33.340151800 (9h46m32.929496787s) [upload=10181]: Starting upload Aug 7 19:26:33.340236143 (9h46m32.92958118s) [upload=10181]: Done starting upload Aug 7 19:26:33.340332376 (9h46m32.929677353s) [upload=10181]: Uploading segments... Aug 7 19:26:33.340373115 (9h46m32.929718112s) [upload=10181]: Got next segment. Inline: true Aug 7 19:26:33.340377673 (9h46m32.92972266s) [upload=10181]: Waiting for error group managing segments... Aug 7 19:26:33.340410516 (9h46m32.929755503s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:26:33.340414664 (9h46m32.929759651s) [upload=10181]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:26:33.340416918 (9h46m32.929761905s) [upload=10181]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:26:33.616120797 (9h46m33.205465824s) [upload=10181]: Done uploading segments... Aug 7 19:26:33.616142689 (9h46m33.205487666s) [upload=10181]: Upload finished. err: <nil> Aug 7 19:27:09.886832212 (9h47m9.476177199s) [upload=10193]: Starting upload Aug 7 19:27:09.886900242 (9h47m9.476245229s) [upload=10193]: Done starting upload Aug 7 19:27:09.886966249 (9h47m9.476311236s) [upload=10193]: Uploading segments... Aug 7 19:27:09.887030491 (9h47m9.476375478s) [upload=10193]: Got next segment. Inline: true Aug 7 19:27:09.887034268 (9h47m9.476379255s) [upload=10193]: Waiting for error group managing segments... Aug 7 19:27:09.887069426 (9h47m9.476414413s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:09.887098522 (9h47m9.476443509s) [upload=10193]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:09.887100906 (9h47m9.476445893s) [upload=10193]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:10.160678358 (9h47m9.750023345s) [upload=10193]: Done uploading segments... Aug 7 19:27:10.160694838 (9h47m9.750039826s) [upload=10193]: Upload finished. err: <nil> Aug 7 19:27:51.977227626 (9h47m51.566572613s) [upload=10208]: Starting upload Aug 7 19:27:51.977292090 (9h47m51.566637067s) [upload=10208]: Done starting upload Aug 7 19:27:51.977366050 (9h47m51.566711037s) [upload=10208]: Uploading segments... Aug 7 19:27:51.977404835 (9h47m51.566749822s) [upload=10208]: Got next segment. Inline: true Aug 7 19:27:51.977408833 (9h47m51.56675382s) [upload=10208]: Waiting for error group managing segments... Aug 7 19:27:51.977441455 (9h47m51.566786442s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:27:51.977445262 (9h47m51.566790249s) [upload=10208]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 19:27:51.977447556 (9h47m51.566792543s) [upload=10208]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:27:52.192621107 (9h47m51.781966094s) [upload=10208]: Done uploading segments... Aug 7 19:27:52.192637719 (9h47m51.781982706s) [upload=10208]: Upload finished. err: metaclient: Too Many Requests Duration: 868.450181ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................X........X.............................] duration: 5.762417276s FAIL (2025-08-07 19:27:22 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.762452403s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.862620225s FAIL (2025-08-07 19:27:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.86264474s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................X........X............................] duration: 6.304783884s FAIL (2025-08-07 19:26:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 6.304809403s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.856537589s FAIL (2025-08-07 19:26:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.856581794s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [X......................X........X...........................] duration: 5.989786321s FAIL (2025-08-07 19:25:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.989811769s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X......................X........X..........................] duration: 5.66866779s FAIL (2025-08-07 19:25:32 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.668736533s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.815727728s FAIL (2025-08-07 19:25:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.815773897s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.926375323s FAIL (2025-08-07 19:24:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.92640527s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X......................X........X.........................] duration: 6.489780555s FAIL (2025-08-07 19:24:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 6.489808468s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.115793639s FAIL (2025-08-07 19:24:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 2.11582527s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X......................X........X........................] duration: 6.033021661s FAIL (2025-08-07 19:24:15 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 6.033048563s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X......X....................................................] duration: 1.846730163s FAIL (2025-08-07 19:23:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.85055465s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X......................X........X.......................] duration: 5.824471919s FAIL (2025-08-07 19:23:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.82449292s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X......................X........X......................] duration: 5.994823185s FAIL (2025-08-07 19:22:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.994854395s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X......X...................................................] duration: 2.012559394s FAIL (2025-08-07 19:22:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 2.012578592s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X......................X........X.....................] duration: 6.342284825s FAIL (2025-08-07 19:21:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 6.342310364s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X......X..................................................] duration: 1.833679074s FAIL (2025-08-07 19:21:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.833705144s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X......................X........X....................] duration: 5.799848507s FAIL (2025-08-07 19:21:00 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.799869598s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X......X.................................................] duration: 1.986702253s FAIL (2025-08-07 19:20:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.986729314s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X......................X........X...................] duration: 6.71623822s FAIL (2025-08-07 19:19:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 6.716264951s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)