Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [......X................X......................X........X....] duration: 6.633280976s FAIL (2025-08-07 19:06:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 6.633306765s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..X..X.............X......X.................................] duration: 1.882276263s FAIL (2025-08-07 19:06:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.882301622s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...X..X.............X......X................................] duration: 1.931467792s FAIL (2025-08-07 19:06:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.931492389s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [....X..X.............X......X...............................] duration: 1.853468758s FAIL (2025-08-07 19:06:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.853492084s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......X................X......................X........X...] duration: 6.192459835s FAIL (2025-08-07 19:06:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 6.192491395s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [........X................X......................X........X..] duration: 5.90063697s FAIL (2025-08-07 19:05:42 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 5.900675212s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........X................X......................X........X.] duration: 6.334694809s FAIL (2025-08-07 19:05:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 6.334714235s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.....X..X.............X......X..............................] duration: 2.132407435s FAIL (2025-08-07 19:05:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 2.132433925s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........X................X......................X........X] duration: 4.383294009s FAIL (2025-08-07 19:04:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== =9811]: Uploading segments... Aug 7 19:04:12.273931080 (9h24m11.863276067s) [upload=9811]: Got next segment. Inline: false Aug 7 19:04:12.274006835 (9h24m11.863351823s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 19:04:12.274011725 (9h24m11.863356702s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 19:04:12.717864015 (9h24m12.307209002s) [seg_pos={0 0}, upload=9811]: Begin upload segment... Aug 7 19:04:12.717911636 (9h24m12.307256623s) [seg_pos={0 0}, upload=9811]: Done begin upload segment. Aug 7 19:04:12.717932836 (9h24m12.307277823s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.717971501 (9h24m12.307316498s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718456990 (9h24m12.307801977s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718545539 (9h24m12.307890526s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718798202 (9h24m12.308143189s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.718904957 (9h24m12.308249944s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719112313 (9h24m12.3084573s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719182157 (9h24m12.308527144s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719442745 (9h24m12.308787722s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719458175 (9h24m12.308803162s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719720877 (9h24m12.309065864s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.719805901 (9h24m12.309150908s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720124269 (9h24m12.309469256s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720266952 (9h24m12.309611939s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720459661 (9h24m12.309804648s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720607645 (9h24m12.309952632s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.720811244 (9h24m12.310156231s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:12.721050341 (9h24m12.310395329s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Uploading piece... Aug 7 19:04:13.779417755 (9h24m13.368762733s) [upload=9811]: Next returned nil segment Aug 7 19:04:13.779436351 (9h24m13.368781338s) [upload=9811]: Waiting for error group managing segments... Aug 7 19:04:14.170293026 (9h24m13.759638013s) [noise=true , address=79.127.205.234:4599 , piece_id=SIEAB3SBKO5DMWUQK65XIAG5X3OLVLTZQYWRWONYPC3FU6PMUFLQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.633221203 (9h24m14.22256619s) [noise=true , address=104.36.82.2:4736 , piece_id=45J7CVQM4OI6BOTIFQSUXHIVMJ3K3LKI4I4YEM7THANMFR7TPBFQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.644767184 (9h24m14.234112171s) [noise=true , address=104.36.82.23:4453 , piece_id=7ETC3JOZLBQWLPLD7V5CEAKUTN2NHCEIWOXG2X2MBKGS7PGT5BGQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.724454327 (9h24m14.313799304s) [noise=true , address=104.36.82.46:4502 , piece_id=OASXKJZFE2JJWBJAZGIFJB5U3RNIFHS3Q2IW7E2GXE6RV5DYA6HA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.735198443 (9h24m14.32454342s) [noise=true , address=104.36.82.35:4589 , piece_id=3ZFRO4FATGT6ILXHWGZNGZIZRJQIAATMEK73D6CJHLMDWRKKXP4A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.768653773 (9h24m14.35799875s) [noise=true , address=104.36.82.59:4308 , piece_id=JB3HUWNCM2BPE4IAUEJXM7DZTKXVKH3BSUFL4WYJUOPEEKKWYTXQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.782851717 (9h24m14.372196704s) [noise=true , address=104.36.82.4:4831 , piece_id=J7WI3SK3EV3VJTQVJFIRWNST5GDDRVEWDYSF5CKWAT2R4VZFHYVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.795000971 (9h24m14.384346009s) [noise=true , address=104.36.82.58:4279 , piece_id=DRVGX3U2RVWWTDWTBNIRMBVJP4MWGCJKQ4CJNSXXH66WJUSEKIXA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.822994261 (9h24m14.412339248s) [noise=true , address=104.36.82.44:4870 , piece_id=YWYYRXU2WE7JU5W2SN2WC2NPMMCWDWWKCBCO4ID2OTXQJ7JFYXVA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.828488021 (9h24m14.417833008s) [noise=true , address=104.36.82.53:4142 , piece_id=HWSPUPW5ESCUNESSIPDTNOL5UCCS43AXROB5CJ65J2JUYUPJAD3A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.835064614 (9h24m14.424409601s) [noise=true , address=104.36.82.17:4348 , piece_id=CS6OSUTFQZ5OTV5JKHTYIBR3GM7K6KGZSVQBQS6WWPAGPR5UCEGA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.865975906 (9h24m14.455320923s) [noise=true , address=104.36.82.16:4324 , piece_id=PBXPML2PLS7AXTRLEHXMUW47UQAXINKRXLRBDNHAWAXMU7EFB5AA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:14.990330359 (9h24m14.579675346s) [noise=true , address=104.36.82.60:4363 , piece_id=TJXC7IVJHTTON3LCHSMEHBTJYB6PCLKKGFNK6OQAJIBX5YVLJ64A, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206641502 (9h24m14.795986489s) [noise=true , address=45.91.116.7:4031 , piece_id=JO625IB7HL5BSH7LCYOOV6F54MZNBZF2UTWWHVWE6WB5JTDEASMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: <nil> Aug 7 19:04:15.206675617 (9h24m14.796020614s) [seg_pos={0 0}, upload=9811]: Segment reached optimal threshold of 14 pieces. Aug 7 19:04:15.206994617 (9h24m14.796339614s) [noise=true , address=104.36.82.5:4866 , piece_id=PJMGUWJF765C3G2UEN3JAM5HTVLYJI73SOBCUDX7AE3CMFJBZPMQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58036->104.36.82.5:4866: use of closed network connection Aug 7 19:04:15.207030646 (9h24m14.796375633s) [noise=true , address=104.36.82.43:4845 , piece_id=SHA54UGBQXHXTBEYIFE3SLLERZAIEDF4AK6G3S7YXOIU3PBXEPDQ, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bgD3PwFrAz1UmQvd8V2rNAf7dFMbjrUEvn1ATTpSM85HijKHS): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207219327 (9h24m14.796564314s) [noise=true , address=104.36.82.15:4267 , piece_id=6Y627WAZEB7FQ6FVLSOBHZY445UWPPVTCXDU4WNBWWKDI63O2UNA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 7 19:04:15.207229436 (9h24m14.796574443s) [noise=true , address=104.36.82.62:4409 , piece_id=XZTRAXD5DC3XAE3PWDVBLTJ4AW6TOU4XD5LZDWMGFE42AI42P2GA, seg_pos={0 0}, upload=9811]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UoCGA4FjE4yADPXL1oifSPJssPgSGh81REZHwGv2eu1Y1kEaj): protocol: read tcp 188.166.225.170:44866->104.36.82.62:4409: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 19:04:15.207247130 (9h24m14.796592127s) [seg_pos={0 0}, upload=9811]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 19:04:15.207332914 (9h24m14.796677911s) [upload=9811]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 19:04:15.207338064 (9h24m14.796683051s) [upload=9811]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 19:04:15.559132112 (9h24m15.148477109s) [upload=9811]: Done uploading segments... Aug 7 19:04:15.559205572 (9h24m15.148550599s) [upload=9811]: Upload finished. err: <nil> Duration: 4.383327713s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [......X..X.............X......X.............................] duration: 1.829556469s FAIL (2025-08-07 19:04:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.829581777s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.......X..X.............X......X............................] duration: 1.866529648s FAIL (2025-08-07 19:03:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.866558924s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X................X......................X........] duration: 5.841916286s FAIL (2025-08-07 19:03:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c9164f94c6ec0d8be03bd7add561fe21cabedef6033940fe48807dbde0c7e925 != before 295cce57eddd82c41c95f775ecb221bf8e5adeebb7e0f92d2a266d351dc89ac3 ======LOGS====== [upload=9681]: Done uploading segments... Aug 7 18:56:38.026009666 (9h16m37.615354653s) [upload=9681]: Upload finished. err: <nil> Aug 7 18:57:15.407740061 (9h17m14.997085038s) [upload=9692]: Starting upload Aug 7 18:57:15.407788083 (9h17m14.99713307s) [upload=9692]: Done starting upload Aug 7 18:57:15.407925035 (9h17m14.997270033s) [upload=9692]: Uploading segments... Aug 7 18:57:15.408041919 (9h17m14.997386906s) [upload=9692]: Got next segment. Inline: false Aug 7 18:57:15.408074691 (9h17m14.997419668s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:57:15.408079991 (9h17m14.997424978s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:57:15.863746153 (9h17m15.453091181s) [seg_pos={0 0}, upload=9692]: Begin upload segment... Aug 7 18:57:15.863818231 (9h17m15.453163258s) [seg_pos={0 0}, upload=9692]: Done begin upload segment. Aug 7 18:57:15.863886011 (9h17m15.453230988s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.863910438 (9h17m15.453255455s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864336432 (9h17m15.453681419s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864476430 (9h17m15.453821417s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864681032 (9h17m15.454026019s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864768960 (9h17m15.454113947s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864966999 (9h17m15.454311986s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865052773 (9h17m15.45439776s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865245011 (9h17m15.454589998s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865338890 (9h17m15.454683877s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865494358 (9h17m15.454839345s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865570824 (9h17m15.454915811s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865723806 (9h17m15.455068783s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865843476 (9h17m15.455188463s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866078325 (9h17m15.455423312s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866293236 (9h17m15.455638223s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876143683 (9h17m15.46548867s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876572673 (9h17m15.46591766s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:16.981893897 (9h17m16.571238884s) [upload=9692]: Next returned nil segment Aug 7 18:57:16.981910720 (9h17m16.571255707s) [upload=9692]: Waiting for error group managing segments... Aug 7 18:57:17.282383809 (9h17m16.871728786s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.362686217 (9h17m16.952031204s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.467201821 (9h17m17.056546798s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.780358728 (9h17m17.369703715s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.784872915 (9h17m17.374217902s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.809758587 (9h17m17.399103574s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858515671 (9h17m17.447860658s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858971273 (9h17m17.44831626s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.862240016 (9h17m17.451585003s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.892544998 (9h17m17.481889985s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.894351945 (9h17m17.483696932s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.905830877 (9h17m17.495175864s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.908331170 (9h17m17.497676157s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941224224 (9h17m17.530569201s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941249463 (9h17m17.53059445s) [seg_pos={0 0}, upload=9692]: Segment reached optimal threshold of 14 pieces. Aug 7 18:57:17.941365675 (9h17m17.530710652s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BCJsPfe1NNUxVUfQWKbvkzviQWYgMxkRZjDhKJ7KFZc1GmnjV): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941420339 (9h17m17.530765326s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941472990 (9h17m17.530817977s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941487898 (9h17m17.530832875s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941535540 (9h17m17.530880527s) [seg_pos={0 0}, upload=9692]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:57:17.941621384 (9h17m17.530966371s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:57:17.941626083 (9h17m17.53097107s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:57:18.228536803 (9h17m17.81788179s) [upload=9692]: Done uploading segments... Aug 7 18:57:18.228554086 (9h17m17.817899073s) [upload=9692]: Upload finished. err: <nil> Duration: 5.841941807s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [........X..X.............X......X...........................] duration: 2.038464336s FAIL (2025-08-07 19:02:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 2.038486439s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X................X......................X.......] duration: 6.277402273s FAIL (2025-08-07 19:02:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c9164f94c6ec0d8be03bd7add561fe21cabedef6033940fe48807dbde0c7e925 != before 295cce57eddd82c41c95f775ecb221bf8e5adeebb7e0f92d2a266d351dc89ac3 ======LOGS====== [upload=9681]: Done uploading segments... Aug 7 18:56:38.026009666 (9h16m37.615354653s) [upload=9681]: Upload finished. err: <nil> Aug 7 18:57:15.407740061 (9h17m14.997085038s) [upload=9692]: Starting upload Aug 7 18:57:15.407788083 (9h17m14.99713307s) [upload=9692]: Done starting upload Aug 7 18:57:15.407925035 (9h17m14.997270033s) [upload=9692]: Uploading segments... Aug 7 18:57:15.408041919 (9h17m14.997386906s) [upload=9692]: Got next segment. Inline: false Aug 7 18:57:15.408074691 (9h17m14.997419668s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:57:15.408079991 (9h17m14.997424978s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:57:15.863746153 (9h17m15.453091181s) [seg_pos={0 0}, upload=9692]: Begin upload segment... Aug 7 18:57:15.863818231 (9h17m15.453163258s) [seg_pos={0 0}, upload=9692]: Done begin upload segment. Aug 7 18:57:15.863886011 (9h17m15.453230988s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.863910438 (9h17m15.453255455s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864336432 (9h17m15.453681419s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864476430 (9h17m15.453821417s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864681032 (9h17m15.454026019s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864768960 (9h17m15.454113947s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864966999 (9h17m15.454311986s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865052773 (9h17m15.45439776s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865245011 (9h17m15.454589998s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865338890 (9h17m15.454683877s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865494358 (9h17m15.454839345s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865570824 (9h17m15.454915811s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865723806 (9h17m15.455068783s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865843476 (9h17m15.455188463s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866078325 (9h17m15.455423312s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866293236 (9h17m15.455638223s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876143683 (9h17m15.46548867s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876572673 (9h17m15.46591766s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:16.981893897 (9h17m16.571238884s) [upload=9692]: Next returned nil segment Aug 7 18:57:16.981910720 (9h17m16.571255707s) [upload=9692]: Waiting for error group managing segments... Aug 7 18:57:17.282383809 (9h17m16.871728786s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.362686217 (9h17m16.952031204s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.467201821 (9h17m17.056546798s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.780358728 (9h17m17.369703715s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.784872915 (9h17m17.374217902s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.809758587 (9h17m17.399103574s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858515671 (9h17m17.447860658s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858971273 (9h17m17.44831626s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.862240016 (9h17m17.451585003s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.892544998 (9h17m17.481889985s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.894351945 (9h17m17.483696932s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.905830877 (9h17m17.495175864s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.908331170 (9h17m17.497676157s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941224224 (9h17m17.530569201s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941249463 (9h17m17.53059445s) [seg_pos={0 0}, upload=9692]: Segment reached optimal threshold of 14 pieces. Aug 7 18:57:17.941365675 (9h17m17.530710652s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BCJsPfe1NNUxVUfQWKbvkzviQWYgMxkRZjDhKJ7KFZc1GmnjV): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941420339 (9h17m17.530765326s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941472990 (9h17m17.530817977s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941487898 (9h17m17.530832875s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941535540 (9h17m17.530880527s) [seg_pos={0 0}, upload=9692]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:57:17.941621384 (9h17m17.530966371s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:57:17.941626083 (9h17m17.53097107s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:57:18.228536803 (9h17m17.81788179s) [upload=9692]: Done uploading segments... Aug 7 18:57:18.228554086 (9h17m17.817899073s) [upload=9692]: Upload finished. err: <nil> Duration: 6.27743217s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X................X......................X......] duration: 5.979275947s FAIL (2025-08-07 19:01:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c9164f94c6ec0d8be03bd7add561fe21cabedef6033940fe48807dbde0c7e925 != before 295cce57eddd82c41c95f775ecb221bf8e5adeebb7e0f92d2a266d351dc89ac3 ======LOGS====== [upload=9681]: Done uploading segments... Aug 7 18:56:38.026009666 (9h16m37.615354653s) [upload=9681]: Upload finished. err: <nil> Aug 7 18:57:15.407740061 (9h17m14.997085038s) [upload=9692]: Starting upload Aug 7 18:57:15.407788083 (9h17m14.99713307s) [upload=9692]: Done starting upload Aug 7 18:57:15.407925035 (9h17m14.997270033s) [upload=9692]: Uploading segments... Aug 7 18:57:15.408041919 (9h17m14.997386906s) [upload=9692]: Got next segment. Inline: false Aug 7 18:57:15.408074691 (9h17m14.997419668s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:57:15.408079991 (9h17m14.997424978s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:57:15.863746153 (9h17m15.453091181s) [seg_pos={0 0}, upload=9692]: Begin upload segment... Aug 7 18:57:15.863818231 (9h17m15.453163258s) [seg_pos={0 0}, upload=9692]: Done begin upload segment. Aug 7 18:57:15.863886011 (9h17m15.453230988s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.863910438 (9h17m15.453255455s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864336432 (9h17m15.453681419s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864476430 (9h17m15.453821417s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864681032 (9h17m15.454026019s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864768960 (9h17m15.454113947s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864966999 (9h17m15.454311986s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865052773 (9h17m15.45439776s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865245011 (9h17m15.454589998s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865338890 (9h17m15.454683877s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865494358 (9h17m15.454839345s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865570824 (9h17m15.454915811s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865723806 (9h17m15.455068783s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865843476 (9h17m15.455188463s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866078325 (9h17m15.455423312s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866293236 (9h17m15.455638223s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876143683 (9h17m15.46548867s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876572673 (9h17m15.46591766s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:16.981893897 (9h17m16.571238884s) [upload=9692]: Next returned nil segment Aug 7 18:57:16.981910720 (9h17m16.571255707s) [upload=9692]: Waiting for error group managing segments... Aug 7 18:57:17.282383809 (9h17m16.871728786s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.362686217 (9h17m16.952031204s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.467201821 (9h17m17.056546798s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.780358728 (9h17m17.369703715s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.784872915 (9h17m17.374217902s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.809758587 (9h17m17.399103574s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858515671 (9h17m17.447860658s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858971273 (9h17m17.44831626s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.862240016 (9h17m17.451585003s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.892544998 (9h17m17.481889985s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.894351945 (9h17m17.483696932s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.905830877 (9h17m17.495175864s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.908331170 (9h17m17.497676157s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941224224 (9h17m17.530569201s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941249463 (9h17m17.53059445s) [seg_pos={0 0}, upload=9692]: Segment reached optimal threshold of 14 pieces. Aug 7 18:57:17.941365675 (9h17m17.530710652s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BCJsPfe1NNUxVUfQWKbvkzviQWYgMxkRZjDhKJ7KFZc1GmnjV): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941420339 (9h17m17.530765326s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941472990 (9h17m17.530817977s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941487898 (9h17m17.530832875s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941535540 (9h17m17.530880527s) [seg_pos={0 0}, upload=9692]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:57:17.941621384 (9h17m17.530966371s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:57:17.941626083 (9h17m17.53097107s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:57:18.228536803 (9h17m17.81788179s) [upload=9692]: Done uploading segments... Aug 7 18:57:18.228554086 (9h17m17.817899073s) [upload=9692]: Upload finished. err: <nil> Duration: 5.979302096s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.........X..X.............X......X..........................] duration: 2.08829322s FAIL (2025-08-07 19:01:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 2.088322207s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X................X......................X.....] duration: 6.018630677s FAIL (2025-08-07 19:00:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c9164f94c6ec0d8be03bd7add561fe21cabedef6033940fe48807dbde0c7e925 != before 295cce57eddd82c41c95f775ecb221bf8e5adeebb7e0f92d2a266d351dc89ac3 ======LOGS====== [upload=9681]: Done uploading segments... Aug 7 18:56:38.026009666 (9h16m37.615354653s) [upload=9681]: Upload finished. err: <nil> Aug 7 18:57:15.407740061 (9h17m14.997085038s) [upload=9692]: Starting upload Aug 7 18:57:15.407788083 (9h17m14.99713307s) [upload=9692]: Done starting upload Aug 7 18:57:15.407925035 (9h17m14.997270033s) [upload=9692]: Uploading segments... Aug 7 18:57:15.408041919 (9h17m14.997386906s) [upload=9692]: Got next segment. Inline: false Aug 7 18:57:15.408074691 (9h17m14.997419668s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:57:15.408079991 (9h17m14.997424978s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:57:15.863746153 (9h17m15.453091181s) [seg_pos={0 0}, upload=9692]: Begin upload segment... Aug 7 18:57:15.863818231 (9h17m15.453163258s) [seg_pos={0 0}, upload=9692]: Done begin upload segment. Aug 7 18:57:15.863886011 (9h17m15.453230988s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.863910438 (9h17m15.453255455s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864336432 (9h17m15.453681419s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864476430 (9h17m15.453821417s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864681032 (9h17m15.454026019s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864768960 (9h17m15.454113947s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864966999 (9h17m15.454311986s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865052773 (9h17m15.45439776s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865245011 (9h17m15.454589998s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865338890 (9h17m15.454683877s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865494358 (9h17m15.454839345s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865570824 (9h17m15.454915811s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865723806 (9h17m15.455068783s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865843476 (9h17m15.455188463s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866078325 (9h17m15.455423312s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866293236 (9h17m15.455638223s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876143683 (9h17m15.46548867s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876572673 (9h17m15.46591766s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:16.981893897 (9h17m16.571238884s) [upload=9692]: Next returned nil segment Aug 7 18:57:16.981910720 (9h17m16.571255707s) [upload=9692]: Waiting for error group managing segments... Aug 7 18:57:17.282383809 (9h17m16.871728786s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.362686217 (9h17m16.952031204s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.467201821 (9h17m17.056546798s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.780358728 (9h17m17.369703715s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.784872915 (9h17m17.374217902s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.809758587 (9h17m17.399103574s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858515671 (9h17m17.447860658s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858971273 (9h17m17.44831626s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.862240016 (9h17m17.451585003s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.892544998 (9h17m17.481889985s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.894351945 (9h17m17.483696932s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.905830877 (9h17m17.495175864s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.908331170 (9h17m17.497676157s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941224224 (9h17m17.530569201s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941249463 (9h17m17.53059445s) [seg_pos={0 0}, upload=9692]: Segment reached optimal threshold of 14 pieces. Aug 7 18:57:17.941365675 (9h17m17.530710652s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BCJsPfe1NNUxVUfQWKbvkzviQWYgMxkRZjDhKJ7KFZc1GmnjV): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941420339 (9h17m17.530765326s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941472990 (9h17m17.530817977s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941487898 (9h17m17.530832875s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941535540 (9h17m17.530880527s) [seg_pos={0 0}, upload=9692]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:57:17.941621384 (9h17m17.530966371s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:57:17.941626083 (9h17m17.53097107s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:57:18.228536803 (9h17m17.81788179s) [upload=9692]: Done uploading segments... Aug 7 18:57:18.228554086 (9h17m17.817899073s) [upload=9692]: Upload finished. err: <nil> Duration: 6.018682976s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..........X..X.............X......X.........................] duration: 1.869498778s FAIL (2025-08-07 19:00:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.869530248s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X................X......................X....] duration: 5.712202703s FAIL (2025-08-07 18:59:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c9164f94c6ec0d8be03bd7add561fe21cabedef6033940fe48807dbde0c7e925 != before 295cce57eddd82c41c95f775ecb221bf8e5adeebb7e0f92d2a266d351dc89ac3 ======LOGS====== [upload=9681]: Done uploading segments... Aug 7 18:56:38.026009666 (9h16m37.615354653s) [upload=9681]: Upload finished. err: <nil> Aug 7 18:57:15.407740061 (9h17m14.997085038s) [upload=9692]: Starting upload Aug 7 18:57:15.407788083 (9h17m14.99713307s) [upload=9692]: Done starting upload Aug 7 18:57:15.407925035 (9h17m14.997270033s) [upload=9692]: Uploading segments... Aug 7 18:57:15.408041919 (9h17m14.997386906s) [upload=9692]: Got next segment. Inline: false Aug 7 18:57:15.408074691 (9h17m14.997419668s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:57:15.408079991 (9h17m14.997424978s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:57:15.863746153 (9h17m15.453091181s) [seg_pos={0 0}, upload=9692]: Begin upload segment... Aug 7 18:57:15.863818231 (9h17m15.453163258s) [seg_pos={0 0}, upload=9692]: Done begin upload segment. Aug 7 18:57:15.863886011 (9h17m15.453230988s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.863910438 (9h17m15.453255455s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864336432 (9h17m15.453681419s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864476430 (9h17m15.453821417s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864681032 (9h17m15.454026019s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864768960 (9h17m15.454113947s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864966999 (9h17m15.454311986s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865052773 (9h17m15.45439776s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865245011 (9h17m15.454589998s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865338890 (9h17m15.454683877s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865494358 (9h17m15.454839345s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865570824 (9h17m15.454915811s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865723806 (9h17m15.455068783s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865843476 (9h17m15.455188463s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866078325 (9h17m15.455423312s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866293236 (9h17m15.455638223s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876143683 (9h17m15.46548867s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876572673 (9h17m15.46591766s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:16.981893897 (9h17m16.571238884s) [upload=9692]: Next returned nil segment Aug 7 18:57:16.981910720 (9h17m16.571255707s) [upload=9692]: Waiting for error group managing segments... Aug 7 18:57:17.282383809 (9h17m16.871728786s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.362686217 (9h17m16.952031204s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.467201821 (9h17m17.056546798s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.780358728 (9h17m17.369703715s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.784872915 (9h17m17.374217902s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.809758587 (9h17m17.399103574s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858515671 (9h17m17.447860658s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858971273 (9h17m17.44831626s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.862240016 (9h17m17.451585003s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.892544998 (9h17m17.481889985s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.894351945 (9h17m17.483696932s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.905830877 (9h17m17.495175864s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.908331170 (9h17m17.497676157s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941224224 (9h17m17.530569201s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941249463 (9h17m17.53059445s) [seg_pos={0 0}, upload=9692]: Segment reached optimal threshold of 14 pieces. Aug 7 18:57:17.941365675 (9h17m17.530710652s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BCJsPfe1NNUxVUfQWKbvkzviQWYgMxkRZjDhKJ7KFZc1GmnjV): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941420339 (9h17m17.530765326s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941472990 (9h17m17.530817977s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941487898 (9h17m17.530832875s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941535540 (9h17m17.530880527s) [seg_pos={0 0}, upload=9692]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:57:17.941621384 (9h17m17.530966371s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:57:17.941626083 (9h17m17.53097107s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:57:18.228536803 (9h17m17.81788179s) [upload=9692]: Done uploading segments... Aug 7 18:57:18.228554086 (9h17m17.817899073s) [upload=9692]: Upload finished. err: <nil> Duration: 5.712224855s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...........X..X.............X......X........................] duration: 1.863233959s FAIL (2025-08-07 18:59:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.863260579s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [............X..X.............X......X.......................] duration: 2.107610892s FAIL (2025-08-07 18:58:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 2.107644597s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X................X......................X...] duration: 6.443119476s FAIL (2025-08-07 18:58:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c9164f94c6ec0d8be03bd7add561fe21cabedef6033940fe48807dbde0c7e925 != before 295cce57eddd82c41c95f775ecb221bf8e5adeebb7e0f92d2a266d351dc89ac3 ======LOGS====== [upload=9681]: Done uploading segments... Aug 7 18:56:38.026009666 (9h16m37.615354653s) [upload=9681]: Upload finished. err: <nil> Aug 7 18:57:15.407740061 (9h17m14.997085038s) [upload=9692]: Starting upload Aug 7 18:57:15.407788083 (9h17m14.99713307s) [upload=9692]: Done starting upload Aug 7 18:57:15.407925035 (9h17m14.997270033s) [upload=9692]: Uploading segments... Aug 7 18:57:15.408041919 (9h17m14.997386906s) [upload=9692]: Got next segment. Inline: false Aug 7 18:57:15.408074691 (9h17m14.997419668s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:57:15.408079991 (9h17m14.997424978s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:57:15.863746153 (9h17m15.453091181s) [seg_pos={0 0}, upload=9692]: Begin upload segment... Aug 7 18:57:15.863818231 (9h17m15.453163258s) [seg_pos={0 0}, upload=9692]: Done begin upload segment. Aug 7 18:57:15.863886011 (9h17m15.453230988s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.863910438 (9h17m15.453255455s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864336432 (9h17m15.453681419s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864476430 (9h17m15.453821417s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864681032 (9h17m15.454026019s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864768960 (9h17m15.454113947s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864966999 (9h17m15.454311986s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865052773 (9h17m15.45439776s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865245011 (9h17m15.454589998s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865338890 (9h17m15.454683877s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865494358 (9h17m15.454839345s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865570824 (9h17m15.454915811s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865723806 (9h17m15.455068783s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865843476 (9h17m15.455188463s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866078325 (9h17m15.455423312s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866293236 (9h17m15.455638223s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876143683 (9h17m15.46548867s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876572673 (9h17m15.46591766s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:16.981893897 (9h17m16.571238884s) [upload=9692]: Next returned nil segment Aug 7 18:57:16.981910720 (9h17m16.571255707s) [upload=9692]: Waiting for error group managing segments... Aug 7 18:57:17.282383809 (9h17m16.871728786s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.362686217 (9h17m16.952031204s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.467201821 (9h17m17.056546798s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.780358728 (9h17m17.369703715s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.784872915 (9h17m17.374217902s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.809758587 (9h17m17.399103574s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858515671 (9h17m17.447860658s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858971273 (9h17m17.44831626s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.862240016 (9h17m17.451585003s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.892544998 (9h17m17.481889985s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.894351945 (9h17m17.483696932s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.905830877 (9h17m17.495175864s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.908331170 (9h17m17.497676157s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941224224 (9h17m17.530569201s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941249463 (9h17m17.53059445s) [seg_pos={0 0}, upload=9692]: Segment reached optimal threshold of 14 pieces. Aug 7 18:57:17.941365675 (9h17m17.530710652s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BCJsPfe1NNUxVUfQWKbvkzviQWYgMxkRZjDhKJ7KFZc1GmnjV): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941420339 (9h17m17.530765326s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941472990 (9h17m17.530817977s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941487898 (9h17m17.530832875s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941535540 (9h17m17.530880527s) [seg_pos={0 0}, upload=9692]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:57:17.941621384 (9h17m17.530966371s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:57:17.941626083 (9h17m17.53097107s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:57:18.228536803 (9h17m17.81788179s) [upload=9692]: Done uploading segments... Aug 7 18:57:18.228554086 (9h17m17.817899073s) [upload=9692]: Upload finished. err: <nil> Duration: 6.443147561s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.............X..X.............X......X......................] duration: 2.095803141s FAIL (2025-08-07 18:58:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 2.095828459s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X................X......................X..] duration: 5.915314345s FAIL (2025-08-07 18:57:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c9164f94c6ec0d8be03bd7add561fe21cabedef6033940fe48807dbde0c7e925 != before 295cce57eddd82c41c95f775ecb221bf8e5adeebb7e0f92d2a266d351dc89ac3 ======LOGS====== [upload=9681]: Done uploading segments... Aug 7 18:56:38.026009666 (9h16m37.615354653s) [upload=9681]: Upload finished. err: <nil> Aug 7 18:57:15.407740061 (9h17m14.997085038s) [upload=9692]: Starting upload Aug 7 18:57:15.407788083 (9h17m14.99713307s) [upload=9692]: Done starting upload Aug 7 18:57:15.407925035 (9h17m14.997270033s) [upload=9692]: Uploading segments... Aug 7 18:57:15.408041919 (9h17m14.997386906s) [upload=9692]: Got next segment. Inline: false Aug 7 18:57:15.408074691 (9h17m14.997419668s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:57:15.408079991 (9h17m14.997424978s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:57:15.863746153 (9h17m15.453091181s) [seg_pos={0 0}, upload=9692]: Begin upload segment... Aug 7 18:57:15.863818231 (9h17m15.453163258s) [seg_pos={0 0}, upload=9692]: Done begin upload segment. Aug 7 18:57:15.863886011 (9h17m15.453230988s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.863910438 (9h17m15.453255455s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864336432 (9h17m15.453681419s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864476430 (9h17m15.453821417s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864681032 (9h17m15.454026019s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864768960 (9h17m15.454113947s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864966999 (9h17m15.454311986s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865052773 (9h17m15.45439776s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865245011 (9h17m15.454589998s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865338890 (9h17m15.454683877s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865494358 (9h17m15.454839345s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865570824 (9h17m15.454915811s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865723806 (9h17m15.455068783s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865843476 (9h17m15.455188463s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866078325 (9h17m15.455423312s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866293236 (9h17m15.455638223s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876143683 (9h17m15.46548867s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876572673 (9h17m15.46591766s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:16.981893897 (9h17m16.571238884s) [upload=9692]: Next returned nil segment Aug 7 18:57:16.981910720 (9h17m16.571255707s) [upload=9692]: Waiting for error group managing segments... Aug 7 18:57:17.282383809 (9h17m16.871728786s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.362686217 (9h17m16.952031204s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.467201821 (9h17m17.056546798s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.780358728 (9h17m17.369703715s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.784872915 (9h17m17.374217902s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.809758587 (9h17m17.399103574s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858515671 (9h17m17.447860658s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858971273 (9h17m17.44831626s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.862240016 (9h17m17.451585003s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.892544998 (9h17m17.481889985s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.894351945 (9h17m17.483696932s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.905830877 (9h17m17.495175864s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.908331170 (9h17m17.497676157s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941224224 (9h17m17.530569201s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941249463 (9h17m17.53059445s) [seg_pos={0 0}, upload=9692]: Segment reached optimal threshold of 14 pieces. Aug 7 18:57:17.941365675 (9h17m17.530710652s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BCJsPfe1NNUxVUfQWKbvkzviQWYgMxkRZjDhKJ7KFZc1GmnjV): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941420339 (9h17m17.530765326s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941472990 (9h17m17.530817977s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941487898 (9h17m17.530832875s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941535540 (9h17m17.530880527s) [seg_pos={0 0}, upload=9692]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:57:17.941621384 (9h17m17.530966371s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:57:17.941626083 (9h17m17.53097107s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:57:18.228536803 (9h17m17.81788179s) [upload=9692]: Done uploading segments... Aug 7 18:57:18.228554086 (9h17m17.817899073s) [upload=9692]: Upload finished. err: <nil> Duration: 5.915340977s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X................X......................X.] duration: 5.777888001s FAIL (2025-08-07 18:57:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c9164f94c6ec0d8be03bd7add561fe21cabedef6033940fe48807dbde0c7e925 != before 295cce57eddd82c41c95f775ecb221bf8e5adeebb7e0f92d2a266d351dc89ac3 ======LOGS====== [upload=9681]: Done uploading segments... Aug 7 18:56:38.026009666 (9h16m37.615354653s) [upload=9681]: Upload finished. err: <nil> Aug 7 18:57:15.407740061 (9h17m14.997085038s) [upload=9692]: Starting upload Aug 7 18:57:15.407788083 (9h17m14.99713307s) [upload=9692]: Done starting upload Aug 7 18:57:15.407925035 (9h17m14.997270033s) [upload=9692]: Uploading segments... Aug 7 18:57:15.408041919 (9h17m14.997386906s) [upload=9692]: Got next segment. Inline: false Aug 7 18:57:15.408074691 (9h17m14.997419668s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:57:15.408079991 (9h17m14.997424978s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:57:15.863746153 (9h17m15.453091181s) [seg_pos={0 0}, upload=9692]: Begin upload segment... Aug 7 18:57:15.863818231 (9h17m15.453163258s) [seg_pos={0 0}, upload=9692]: Done begin upload segment. Aug 7 18:57:15.863886011 (9h17m15.453230988s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.863910438 (9h17m15.453255455s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864336432 (9h17m15.453681419s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864476430 (9h17m15.453821417s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864681032 (9h17m15.454026019s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864768960 (9h17m15.454113947s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864966999 (9h17m15.454311986s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865052773 (9h17m15.45439776s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865245011 (9h17m15.454589998s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865338890 (9h17m15.454683877s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865494358 (9h17m15.454839345s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865570824 (9h17m15.454915811s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865723806 (9h17m15.455068783s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865843476 (9h17m15.455188463s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866078325 (9h17m15.455423312s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866293236 (9h17m15.455638223s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876143683 (9h17m15.46548867s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876572673 (9h17m15.46591766s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:16.981893897 (9h17m16.571238884s) [upload=9692]: Next returned nil segment Aug 7 18:57:16.981910720 (9h17m16.571255707s) [upload=9692]: Waiting for error group managing segments... Aug 7 18:57:17.282383809 (9h17m16.871728786s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.362686217 (9h17m16.952031204s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.467201821 (9h17m17.056546798s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.780358728 (9h17m17.369703715s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.784872915 (9h17m17.374217902s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.809758587 (9h17m17.399103574s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858515671 (9h17m17.447860658s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858971273 (9h17m17.44831626s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.862240016 (9h17m17.451585003s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.892544998 (9h17m17.481889985s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.894351945 (9h17m17.483696932s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.905830877 (9h17m17.495175864s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.908331170 (9h17m17.497676157s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941224224 (9h17m17.530569201s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941249463 (9h17m17.53059445s) [seg_pos={0 0}, upload=9692]: Segment reached optimal threshold of 14 pieces. Aug 7 18:57:17.941365675 (9h17m17.530710652s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BCJsPfe1NNUxVUfQWKbvkzviQWYgMxkRZjDhKJ7KFZc1GmnjV): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941420339 (9h17m17.530765326s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941472990 (9h17m17.530817977s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941487898 (9h17m17.530832875s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941535540 (9h17m17.530880527s) [seg_pos={0 0}, upload=9692]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:57:17.941621384 (9h17m17.530966371s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:57:17.941626083 (9h17m17.53097107s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:57:18.228536803 (9h17m17.81788179s) [upload=9692]: Done uploading segments... Aug 7 18:57:18.228554086 (9h17m17.817899073s) [upload=9692]: Upload finished. err: <nil> Duration: 5.777914774s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..............X..X.............X......X.....................] duration: 1.889150963s FAIL (2025-08-07 18:57:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.889177455s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X................X......................X] duration: 5.685577668s FAIL (2025-08-07 18:57:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after c9164f94c6ec0d8be03bd7add561fe21cabedef6033940fe48807dbde0c7e925 != before 295cce57eddd82c41c95f775ecb221bf8e5adeebb7e0f92d2a266d351dc89ac3 ======LOGS====== [upload=9681]: Done uploading segments... Aug 7 18:56:38.026009666 (9h16m37.615354653s) [upload=9681]: Upload finished. err: <nil> Aug 7 18:57:15.407740061 (9h17m14.997085038s) [upload=9692]: Starting upload Aug 7 18:57:15.407788083 (9h17m14.99713307s) [upload=9692]: Done starting upload Aug 7 18:57:15.407925035 (9h17m14.997270033s) [upload=9692]: Uploading segments... Aug 7 18:57:15.408041919 (9h17m14.997386906s) [upload=9692]: Got next segment. Inline: false Aug 7 18:57:15.408074691 (9h17m14.997419668s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:57:15.408079991 (9h17m14.997424978s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:57:15.863746153 (9h17m15.453091181s) [seg_pos={0 0}, upload=9692]: Begin upload segment... Aug 7 18:57:15.863818231 (9h17m15.453163258s) [seg_pos={0 0}, upload=9692]: Done begin upload segment. Aug 7 18:57:15.863886011 (9h17m15.453230988s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.863910438 (9h17m15.453255455s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864336432 (9h17m15.453681419s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864476430 (9h17m15.453821417s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864681032 (9h17m15.454026019s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864768960 (9h17m15.454113947s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.864966999 (9h17m15.454311986s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865052773 (9h17m15.45439776s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865245011 (9h17m15.454589998s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865338890 (9h17m15.454683877s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865494358 (9h17m15.454839345s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865570824 (9h17m15.454915811s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865723806 (9h17m15.455068783s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.865843476 (9h17m15.455188463s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866078325 (9h17m15.455423312s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.866293236 (9h17m15.455638223s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876143683 (9h17m15.46548867s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:15.876572673 (9h17m15.46591766s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Uploading piece... Aug 7 18:57:16.981893897 (9h17m16.571238884s) [upload=9692]: Next returned nil segment Aug 7 18:57:16.981910720 (9h17m16.571255707s) [upload=9692]: Waiting for error group managing segments... Aug 7 18:57:17.282383809 (9h17m16.871728786s) [noise=true , address=64.49.13.170:4728 , piece_id=ZM2DF5KS3RA5ZGLLVE7S2O36KTKGGDWDKH6YRBAG3MGFIKLL66BQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.362686217 (9h17m16.952031204s) [noise=true , address=193.149.164.143:4694 , piece_id=RUHSR5C2BQAFNYWHG7ABEJ2KNPH3Y7IJQBYKHM5ONXJW2CFN2RVA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.467201821 (9h17m17.056546798s) [noise=true , address=79.127.205.233:4633 , piece_id=LDRWRLEPSAJPB3HCA62A5OEOVGH3AADRCIDQBWWCQUFJFB7Z723A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.780358728 (9h17m17.369703715s) [noise=true , address=104.36.82.44:4869 , piece_id=2DIVVCYS775XLUVU2OR3JNQTEG4WCFC4LDJZQ6UM4RIQNAM7MIUQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.784872915 (9h17m17.374217902s) [noise=true , address=104.36.82.6:4879 , piece_id=6VRVDB7L6TKPM5NPMJWOKNO56ZDRJRY4XGAJIWTDWWR7BZOOL3TA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.809758587 (9h17m17.399103574s) [noise=true , address=104.36.82.55:4228 , piece_id=7UQPSZWTWOK33UXGIJ6QMF47IM4U2RZFYSONGZ3QBXQXVDNYLGOQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858515671 (9h17m17.447860658s) [noise=true , address=104.36.82.53:4124 , piece_id=3RBIPEXB4HV4FQMYZDMKUM5CT5UTQKPENQOZ2GRRGIVRSADYTIGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.858971273 (9h17m17.44831626s) [noise=true , address=104.36.82.9:4012 , piece_id=LYZ432SW6JAX2VL5CXIU3CUCFDGGAUP5DVUTSIAVSOHN3PLLXHUA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.862240016 (9h17m17.451585003s) [noise=true , address=45.91.116.7:4020 , piece_id=2Y3ROJZL3UYQWHE64YGUFPO672OFDNJKMMSOZA4G4XP7724BJSCQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.892544998 (9h17m17.481889985s) [noise=true , address=104.36.82.14:4230 , piece_id=AL2SPOCJQHEWYJMLFMN6ZEB46TP753HBJCC4LYL7GVWUMQU24YYA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.894351945 (9h17m17.483696932s) [noise=true , address=104.36.82.59:4314 , piece_id=IX4FZN6W75EQJRZAIXT6JLTR5GKGBJGF3YUSQ6BJQS4RQMGUBAMA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.905830877 (9h17m17.495175864s) [noise=true , address=104.36.82.19:4405 , piece_id=N4M5WO3TEY732XGO64QHDDD6Q3YOPOONYU5HOVO7VXX3EU5BWDBA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.908331170 (9h17m17.497676157s) [noise=true , address=104.36.82.60:4365 , piece_id=JPKZKZJE7T5OI37X2ZHLEHA5I3XCTRMJVLAFAJX4RVCRVLBZUYGQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941224224 (9h17m17.530569201s) [noise=true , address=45.91.116.8:4066 , piece_id=IVLSUVLP6XN5FUOXMNE2VUHVDJAWOW2CAUZD3UF2EPJMVWPBVK7A, seg_pos={0 0}, upload=9692]: Done uploading piece. err: <nil> Aug 7 18:57:17.941249463 (9h17m17.53059445s) [seg_pos={0 0}, upload=9692]: Segment reached optimal threshold of 14 pieces. Aug 7 18:57:17.941365675 (9h17m17.530710652s) [noise=true , address=104.36.82.56:4259 , piece_id=63VJ4HKMCDEX44FU4D4EVVBBTZGNTKDK5W7FXCKKWMQS6XYKMEKQ, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BCJsPfe1NNUxVUfQWKbvkzviQWYgMxkRZjDhKJ7KFZc1GmnjV): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941420339 (9h17m17.530765326s) [noise=true , address=104.36.82.54:4162 , piece_id=ZFJCV5NWFDUZGTIQCRFNHPWRLEQDOOLQGKTPT7HXOMRBXZCLAMXA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941472990 (9h17m17.530817977s) [noise=true , address=104.36.82.23:4455 , piece_id=2QJETADB4XJD2THW5FEMJWGTY4ADHBCT63KLMVX6AYIK367B4WSA, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wsyjF5X3iPrJsezUjakUArxc8K88T36yT4rE3yu6q9BMKfycm): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941487898 (9h17m17.530832875s) [noise=true , address=104.36.82.55:4198 , piece_id=JTHF6QD7RKLK6TOLCY4JMQX2VV4TQKONHX76DFMTZKJ5VLKKLC2Q, seg_pos={0 0}, upload=9692]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YxL4YR6mNE536w1SnBrMUZHiaRTuU68Zjg9LBvRURVx5s5HWL): protocol: expected piece hash; context canceled; EOF Aug 7 18:57:17.941535540 (9h17m17.530880527s) [seg_pos={0 0}, upload=9692]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:57:17.941621384 (9h17m17.530966371s) [upload=9692]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:57:17.941626083 (9h17m17.53097107s) [upload=9692]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:57:18.228536803 (9h17m17.81788179s) [upload=9692]: Done uploading segments... Aug 7 18:57:18.228554086 (9h17m17.817899073s) [upload=9692]: Upload finished. err: <nil> Duration: 5.685608315s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................X................X......................] duration: 5.865736117s FAIL (2025-08-07 18:56:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== m8.735137734s) [upload=9375]: Uploading segments... Aug 7 18:39:09.145944347 (8h59m8.735289335s) [upload=9375]: Got next segment. Inline: false Aug 7 18:39:09.145982490 (8h59m8.735327467s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:09.145986848 (8h59m8.735331835s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:39:09.642154224 (8h59m9.231499201s) [seg_pos={0 0}, upload=9375]: Begin upload segment... Aug 7 18:39:09.642193350 (8h59m9.231538327s) [seg_pos={0 0}, upload=9375]: Done begin upload segment. Aug 7 18:39:09.642214950 (8h59m9.231559937s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642229899 (8h59m9.231574896s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642582825 (8h59m9.231927812s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642865325 (8h59m9.232210302s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643156402 (8h59m9.232501389s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643442810 (8h59m9.232787787s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643676567 (8h59m9.233021554s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643970980 (8h59m9.233315967s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644264642 (8h59m9.23360963s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644537404 (8h59m9.233882392s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644749891 (8h59m9.234094868s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644963820 (8h59m9.234308807s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645264726 (8h59m9.234609713s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645483795 (8h59m9.234828782s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645575301 (8h59m9.234920288s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645847200 (8h59m9.235192177s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646083532 (8h59m9.235428519s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646308663 (8h59m9.23565365s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:10.758983522 (8h59m10.348328509s) [upload=9375]: Next returned nil segment Aug 7 18:39:10.758998891 (8h59m10.348343878s) [upload=9375]: Waiting for error group managing segments... Aug 7 18:39:11.040903333 (8h59m10.63024832s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.575207667 (8h59m11.164552654s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.580126699 (8h59m11.169471696s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.620341280 (8h59m11.209686277s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.641699104 (8h59m11.231044091s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.647426883 (8h59m11.23677187s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.662592328 (8h59m11.251937305s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.697065164 (8h59m11.286410151s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.720412004 (8h59m11.309756991s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.731711513 (8h59m11.3210565s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.743036420 (8h59m11.332381407s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.811114753 (8h59m11.40045974s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.889740355 (8h59m11.479085342s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996286117 (8h59m11.585631104s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996311105 (8h59m11.585656092s) [seg_pos={0 0}, upload=9375]: Segment reached optimal threshold of 14 pieces. Aug 7 18:39:11.996413861 (8h59m11.585758848s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ex8eUvL1Hqxd5EffckSvBeZDpGrMGhKENyCPvqHybtQf3YQzM): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996588316 (8h59m11.585933303s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vucBTonz7hjJgo2Jy2rhpSryqK6ifrBtj34pAcqc8FA5nPyYp): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996656285 (8h59m11.586001272s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123rwT3AmQR5nrvjoo6fgKBBJaKVWJHkJ6FtLnoNYs2AxakoDZC): protocol: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection Aug 7 18:39:11.996741468 (8h59m11.586086455s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ): protocol: read tcp 188.166.225.170:50042->104.36.82.58:4265: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:39:11.996761066 (8h59m11.586106053s) [seg_pos={0 0}, upload=9375]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:39:11.996823245 (8h59m11.586168232s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:39:11.996827292 (8h59m11.586172269s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:12.298790633 (8h59m11.88813562s) [upload=9375]: Done uploading segments... Aug 7 18:39:12.298809429 (8h59m11.888154416s) [upload=9375]: Upload finished. err: <nil> Duration: 5.86576398s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...............X..X.............X......X....................] duration: 1.893754101s FAIL (2025-08-07 18:56:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.893783356s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................X................X.....................] duration: 5.779885813s FAIL (2025-08-07 18:55:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== m8.735137734s) [upload=9375]: Uploading segments... Aug 7 18:39:09.145944347 (8h59m8.735289335s) [upload=9375]: Got next segment. Inline: false Aug 7 18:39:09.145982490 (8h59m8.735327467s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:09.145986848 (8h59m8.735331835s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:39:09.642154224 (8h59m9.231499201s) [seg_pos={0 0}, upload=9375]: Begin upload segment... Aug 7 18:39:09.642193350 (8h59m9.231538327s) [seg_pos={0 0}, upload=9375]: Done begin upload segment. Aug 7 18:39:09.642214950 (8h59m9.231559937s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642229899 (8h59m9.231574896s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642582825 (8h59m9.231927812s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642865325 (8h59m9.232210302s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643156402 (8h59m9.232501389s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643442810 (8h59m9.232787787s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643676567 (8h59m9.233021554s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643970980 (8h59m9.233315967s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644264642 (8h59m9.23360963s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644537404 (8h59m9.233882392s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644749891 (8h59m9.234094868s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644963820 (8h59m9.234308807s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645264726 (8h59m9.234609713s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645483795 (8h59m9.234828782s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645575301 (8h59m9.234920288s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645847200 (8h59m9.235192177s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646083532 (8h59m9.235428519s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646308663 (8h59m9.23565365s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:10.758983522 (8h59m10.348328509s) [upload=9375]: Next returned nil segment Aug 7 18:39:10.758998891 (8h59m10.348343878s) [upload=9375]: Waiting for error group managing segments... Aug 7 18:39:11.040903333 (8h59m10.63024832s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.575207667 (8h59m11.164552654s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.580126699 (8h59m11.169471696s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.620341280 (8h59m11.209686277s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.641699104 (8h59m11.231044091s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.647426883 (8h59m11.23677187s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.662592328 (8h59m11.251937305s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.697065164 (8h59m11.286410151s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.720412004 (8h59m11.309756991s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.731711513 (8h59m11.3210565s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.743036420 (8h59m11.332381407s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.811114753 (8h59m11.40045974s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.889740355 (8h59m11.479085342s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996286117 (8h59m11.585631104s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996311105 (8h59m11.585656092s) [seg_pos={0 0}, upload=9375]: Segment reached optimal threshold of 14 pieces. Aug 7 18:39:11.996413861 (8h59m11.585758848s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ex8eUvL1Hqxd5EffckSvBeZDpGrMGhKENyCPvqHybtQf3YQzM): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996588316 (8h59m11.585933303s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vucBTonz7hjJgo2Jy2rhpSryqK6ifrBtj34pAcqc8FA5nPyYp): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996656285 (8h59m11.586001272s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123rwT3AmQR5nrvjoo6fgKBBJaKVWJHkJ6FtLnoNYs2AxakoDZC): protocol: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection Aug 7 18:39:11.996741468 (8h59m11.586086455s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ): protocol: read tcp 188.166.225.170:50042->104.36.82.58:4265: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:39:11.996761066 (8h59m11.586106053s) [seg_pos={0 0}, upload=9375]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:39:11.996823245 (8h59m11.586168232s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:39:11.996827292 (8h59m11.586172269s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:12.298790633 (8h59m11.88813562s) [upload=9375]: Done uploading segments... Aug 7 18:39:12.298809429 (8h59m11.888154416s) [upload=9375]: Upload finished. err: <nil> Duration: 5.779923236s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [................X..X.............X......X...................] duration: 2.146760574s FAIL (2025-08-07 18:55:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 2.146786273s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................X................X....................] duration: 5.6795722s FAIL (2025-08-07 18:55:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== m8.735137734s) [upload=9375]: Uploading segments... Aug 7 18:39:09.145944347 (8h59m8.735289335s) [upload=9375]: Got next segment. Inline: false Aug 7 18:39:09.145982490 (8h59m8.735327467s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:09.145986848 (8h59m8.735331835s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:39:09.642154224 (8h59m9.231499201s) [seg_pos={0 0}, upload=9375]: Begin upload segment... Aug 7 18:39:09.642193350 (8h59m9.231538327s) [seg_pos={0 0}, upload=9375]: Done begin upload segment. Aug 7 18:39:09.642214950 (8h59m9.231559937s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642229899 (8h59m9.231574896s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642582825 (8h59m9.231927812s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642865325 (8h59m9.232210302s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643156402 (8h59m9.232501389s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643442810 (8h59m9.232787787s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643676567 (8h59m9.233021554s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643970980 (8h59m9.233315967s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644264642 (8h59m9.23360963s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644537404 (8h59m9.233882392s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644749891 (8h59m9.234094868s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644963820 (8h59m9.234308807s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645264726 (8h59m9.234609713s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645483795 (8h59m9.234828782s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645575301 (8h59m9.234920288s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645847200 (8h59m9.235192177s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646083532 (8h59m9.235428519s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646308663 (8h59m9.23565365s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:10.758983522 (8h59m10.348328509s) [upload=9375]: Next returned nil segment Aug 7 18:39:10.758998891 (8h59m10.348343878s) [upload=9375]: Waiting for error group managing segments... Aug 7 18:39:11.040903333 (8h59m10.63024832s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.575207667 (8h59m11.164552654s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.580126699 (8h59m11.169471696s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.620341280 (8h59m11.209686277s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.641699104 (8h59m11.231044091s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.647426883 (8h59m11.23677187s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.662592328 (8h59m11.251937305s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.697065164 (8h59m11.286410151s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.720412004 (8h59m11.309756991s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.731711513 (8h59m11.3210565s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.743036420 (8h59m11.332381407s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.811114753 (8h59m11.40045974s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.889740355 (8h59m11.479085342s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996286117 (8h59m11.585631104s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996311105 (8h59m11.585656092s) [seg_pos={0 0}, upload=9375]: Segment reached optimal threshold of 14 pieces. Aug 7 18:39:11.996413861 (8h59m11.585758848s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ex8eUvL1Hqxd5EffckSvBeZDpGrMGhKENyCPvqHybtQf3YQzM): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996588316 (8h59m11.585933303s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vucBTonz7hjJgo2Jy2rhpSryqK6ifrBtj34pAcqc8FA5nPyYp): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996656285 (8h59m11.586001272s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123rwT3AmQR5nrvjoo6fgKBBJaKVWJHkJ6FtLnoNYs2AxakoDZC): protocol: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection Aug 7 18:39:11.996741468 (8h59m11.586086455s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ): protocol: read tcp 188.166.225.170:50042->104.36.82.58:4265: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:39:11.996761066 (8h59m11.586106053s) [seg_pos={0 0}, upload=9375]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:39:11.996823245 (8h59m11.586168232s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:39:11.996827292 (8h59m11.586172269s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:12.298790633 (8h59m11.88813562s) [upload=9375]: Done uploading segments... Aug 7 18:39:12.298809429 (8h59m11.888154416s) [upload=9375]: Upload finished. err: <nil> Duration: 5.679599062s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................X................X...................] duration: 5.835432754s FAIL (2025-08-07 18:54:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== m8.735137734s) [upload=9375]: Uploading segments... Aug 7 18:39:09.145944347 (8h59m8.735289335s) [upload=9375]: Got next segment. Inline: false Aug 7 18:39:09.145982490 (8h59m8.735327467s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:09.145986848 (8h59m8.735331835s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:39:09.642154224 (8h59m9.231499201s) [seg_pos={0 0}, upload=9375]: Begin upload segment... Aug 7 18:39:09.642193350 (8h59m9.231538327s) [seg_pos={0 0}, upload=9375]: Done begin upload segment. Aug 7 18:39:09.642214950 (8h59m9.231559937s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642229899 (8h59m9.231574896s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642582825 (8h59m9.231927812s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642865325 (8h59m9.232210302s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643156402 (8h59m9.232501389s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643442810 (8h59m9.232787787s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643676567 (8h59m9.233021554s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643970980 (8h59m9.233315967s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644264642 (8h59m9.23360963s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644537404 (8h59m9.233882392s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644749891 (8h59m9.234094868s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644963820 (8h59m9.234308807s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645264726 (8h59m9.234609713s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645483795 (8h59m9.234828782s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645575301 (8h59m9.234920288s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645847200 (8h59m9.235192177s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646083532 (8h59m9.235428519s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646308663 (8h59m9.23565365s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:10.758983522 (8h59m10.348328509s) [upload=9375]: Next returned nil segment Aug 7 18:39:10.758998891 (8h59m10.348343878s) [upload=9375]: Waiting for error group managing segments... Aug 7 18:39:11.040903333 (8h59m10.63024832s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.575207667 (8h59m11.164552654s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.580126699 (8h59m11.169471696s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.620341280 (8h59m11.209686277s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.641699104 (8h59m11.231044091s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.647426883 (8h59m11.23677187s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.662592328 (8h59m11.251937305s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.697065164 (8h59m11.286410151s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.720412004 (8h59m11.309756991s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.731711513 (8h59m11.3210565s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.743036420 (8h59m11.332381407s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.811114753 (8h59m11.40045974s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.889740355 (8h59m11.479085342s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996286117 (8h59m11.585631104s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996311105 (8h59m11.585656092s) [seg_pos={0 0}, upload=9375]: Segment reached optimal threshold of 14 pieces. Aug 7 18:39:11.996413861 (8h59m11.585758848s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ex8eUvL1Hqxd5EffckSvBeZDpGrMGhKENyCPvqHybtQf3YQzM): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996588316 (8h59m11.585933303s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vucBTonz7hjJgo2Jy2rhpSryqK6ifrBtj34pAcqc8FA5nPyYp): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996656285 (8h59m11.586001272s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123rwT3AmQR5nrvjoo6fgKBBJaKVWJHkJ6FtLnoNYs2AxakoDZC): protocol: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection Aug 7 18:39:11.996741468 (8h59m11.586086455s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ): protocol: read tcp 188.166.225.170:50042->104.36.82.58:4265: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:39:11.996761066 (8h59m11.586106053s) [seg_pos={0 0}, upload=9375]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:39:11.996823245 (8h59m11.586168232s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:39:11.996827292 (8h59m11.586172269s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:12.298790633 (8h59m11.88813562s) [upload=9375]: Done uploading segments... Aug 7 18:39:12.298809429 (8h59m11.888154416s) [upload=9375]: Upload finished. err: <nil> Duration: 5.835465437s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.................X..X.............X......X..................] duration: 1.99756214s FAIL (2025-08-07 18:54:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.997609663s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..................X..X.............X......X.................] duration: 2.023228526s FAIL (2025-08-07 18:54:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 2.02325621s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................X................X..................] duration: 5.909236238s FAIL (2025-08-07 18:53:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== m8.735137734s) [upload=9375]: Uploading segments... Aug 7 18:39:09.145944347 (8h59m8.735289335s) [upload=9375]: Got next segment. Inline: false Aug 7 18:39:09.145982490 (8h59m8.735327467s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:09.145986848 (8h59m8.735331835s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:39:09.642154224 (8h59m9.231499201s) [seg_pos={0 0}, upload=9375]: Begin upload segment... Aug 7 18:39:09.642193350 (8h59m9.231538327s) [seg_pos={0 0}, upload=9375]: Done begin upload segment. Aug 7 18:39:09.642214950 (8h59m9.231559937s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642229899 (8h59m9.231574896s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642582825 (8h59m9.231927812s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642865325 (8h59m9.232210302s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643156402 (8h59m9.232501389s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643442810 (8h59m9.232787787s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643676567 (8h59m9.233021554s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643970980 (8h59m9.233315967s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644264642 (8h59m9.23360963s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644537404 (8h59m9.233882392s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644749891 (8h59m9.234094868s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644963820 (8h59m9.234308807s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645264726 (8h59m9.234609713s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645483795 (8h59m9.234828782s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645575301 (8h59m9.234920288s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645847200 (8h59m9.235192177s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646083532 (8h59m9.235428519s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646308663 (8h59m9.23565365s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:10.758983522 (8h59m10.348328509s) [upload=9375]: Next returned nil segment Aug 7 18:39:10.758998891 (8h59m10.348343878s) [upload=9375]: Waiting for error group managing segments... Aug 7 18:39:11.040903333 (8h59m10.63024832s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.575207667 (8h59m11.164552654s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.580126699 (8h59m11.169471696s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.620341280 (8h59m11.209686277s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.641699104 (8h59m11.231044091s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.647426883 (8h59m11.23677187s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.662592328 (8h59m11.251937305s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.697065164 (8h59m11.286410151s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.720412004 (8h59m11.309756991s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.731711513 (8h59m11.3210565s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.743036420 (8h59m11.332381407s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.811114753 (8h59m11.40045974s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.889740355 (8h59m11.479085342s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996286117 (8h59m11.585631104s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996311105 (8h59m11.585656092s) [seg_pos={0 0}, upload=9375]: Segment reached optimal threshold of 14 pieces. Aug 7 18:39:11.996413861 (8h59m11.585758848s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ex8eUvL1Hqxd5EffckSvBeZDpGrMGhKENyCPvqHybtQf3YQzM): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996588316 (8h59m11.585933303s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vucBTonz7hjJgo2Jy2rhpSryqK6ifrBtj34pAcqc8FA5nPyYp): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996656285 (8h59m11.586001272s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123rwT3AmQR5nrvjoo6fgKBBJaKVWJHkJ6FtLnoNYs2AxakoDZC): protocol: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection Aug 7 18:39:11.996741468 (8h59m11.586086455s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ): protocol: read tcp 188.166.225.170:50042->104.36.82.58:4265: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:39:11.996761066 (8h59m11.586106053s) [seg_pos={0 0}, upload=9375]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:39:11.996823245 (8h59m11.586168232s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:39:11.996827292 (8h59m11.586172269s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:12.298790633 (8h59m11.88813562s) [upload=9375]: Done uploading segments... Aug 7 18:39:12.298809429 (8h59m11.888154416s) [upload=9375]: Upload finished. err: <nil> Duration: 5.909285572s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [X..................X..X.............X......X................] duration: 1.826408573s FAIL (2025-08-07 18:52:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.826441356s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.X..................X..X.............X......X...............] duration: 1.834128514s FAIL (2025-08-07 18:52:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.834151498s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................X................X.................] duration: 5.771217078s FAIL (2025-08-07 18:52:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== m8.735137734s) [upload=9375]: Uploading segments... Aug 7 18:39:09.145944347 (8h59m8.735289335s) [upload=9375]: Got next segment. Inline: false Aug 7 18:39:09.145982490 (8h59m8.735327467s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:09.145986848 (8h59m8.735331835s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:39:09.642154224 (8h59m9.231499201s) [seg_pos={0 0}, upload=9375]: Begin upload segment... Aug 7 18:39:09.642193350 (8h59m9.231538327s) [seg_pos={0 0}, upload=9375]: Done begin upload segment. Aug 7 18:39:09.642214950 (8h59m9.231559937s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642229899 (8h59m9.231574896s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642582825 (8h59m9.231927812s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642865325 (8h59m9.232210302s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643156402 (8h59m9.232501389s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643442810 (8h59m9.232787787s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643676567 (8h59m9.233021554s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643970980 (8h59m9.233315967s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644264642 (8h59m9.23360963s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644537404 (8h59m9.233882392s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644749891 (8h59m9.234094868s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644963820 (8h59m9.234308807s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645264726 (8h59m9.234609713s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645483795 (8h59m9.234828782s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645575301 (8h59m9.234920288s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645847200 (8h59m9.235192177s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646083532 (8h59m9.235428519s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646308663 (8h59m9.23565365s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:10.758983522 (8h59m10.348328509s) [upload=9375]: Next returned nil segment Aug 7 18:39:10.758998891 (8h59m10.348343878s) [upload=9375]: Waiting for error group managing segments... Aug 7 18:39:11.040903333 (8h59m10.63024832s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.575207667 (8h59m11.164552654s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.580126699 (8h59m11.169471696s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.620341280 (8h59m11.209686277s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.641699104 (8h59m11.231044091s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.647426883 (8h59m11.23677187s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.662592328 (8h59m11.251937305s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.697065164 (8h59m11.286410151s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.720412004 (8h59m11.309756991s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.731711513 (8h59m11.3210565s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.743036420 (8h59m11.332381407s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.811114753 (8h59m11.40045974s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.889740355 (8h59m11.479085342s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996286117 (8h59m11.585631104s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996311105 (8h59m11.585656092s) [seg_pos={0 0}, upload=9375]: Segment reached optimal threshold of 14 pieces. Aug 7 18:39:11.996413861 (8h59m11.585758848s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ex8eUvL1Hqxd5EffckSvBeZDpGrMGhKENyCPvqHybtQf3YQzM): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996588316 (8h59m11.585933303s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vucBTonz7hjJgo2Jy2rhpSryqK6ifrBtj34pAcqc8FA5nPyYp): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996656285 (8h59m11.586001272s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123rwT3AmQR5nrvjoo6fgKBBJaKVWJHkJ6FtLnoNYs2AxakoDZC): protocol: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection Aug 7 18:39:11.996741468 (8h59m11.586086455s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ): protocol: read tcp 188.166.225.170:50042->104.36.82.58:4265: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:39:11.996761066 (8h59m11.586106053s) [seg_pos={0 0}, upload=9375]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:39:11.996823245 (8h59m11.586168232s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:39:11.996827292 (8h59m11.586172269s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:12.298790633 (8h59m11.88813562s) [upload=9375]: Done uploading segments... Aug 7 18:39:12.298809429 (8h59m11.888154416s) [upload=9375]: Upload finished. err: <nil> Duration: 5.771266053s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................X................X................] duration: 5.769431846s FAIL (2025-08-07 18:51:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== m8.735137734s) [upload=9375]: Uploading segments... Aug 7 18:39:09.145944347 (8h59m8.735289335s) [upload=9375]: Got next segment. Inline: false Aug 7 18:39:09.145982490 (8h59m8.735327467s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:09.145986848 (8h59m8.735331835s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:39:09.642154224 (8h59m9.231499201s) [seg_pos={0 0}, upload=9375]: Begin upload segment... Aug 7 18:39:09.642193350 (8h59m9.231538327s) [seg_pos={0 0}, upload=9375]: Done begin upload segment. Aug 7 18:39:09.642214950 (8h59m9.231559937s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642229899 (8h59m9.231574896s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642582825 (8h59m9.231927812s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642865325 (8h59m9.232210302s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643156402 (8h59m9.232501389s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643442810 (8h59m9.232787787s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643676567 (8h59m9.233021554s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643970980 (8h59m9.233315967s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644264642 (8h59m9.23360963s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644537404 (8h59m9.233882392s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644749891 (8h59m9.234094868s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644963820 (8h59m9.234308807s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645264726 (8h59m9.234609713s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645483795 (8h59m9.234828782s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645575301 (8h59m9.234920288s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645847200 (8h59m9.235192177s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646083532 (8h59m9.235428519s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646308663 (8h59m9.23565365s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:10.758983522 (8h59m10.348328509s) [upload=9375]: Next returned nil segment Aug 7 18:39:10.758998891 (8h59m10.348343878s) [upload=9375]: Waiting for error group managing segments... Aug 7 18:39:11.040903333 (8h59m10.63024832s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.575207667 (8h59m11.164552654s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.580126699 (8h59m11.169471696s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.620341280 (8h59m11.209686277s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.641699104 (8h59m11.231044091s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.647426883 (8h59m11.23677187s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.662592328 (8h59m11.251937305s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.697065164 (8h59m11.286410151s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.720412004 (8h59m11.309756991s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.731711513 (8h59m11.3210565s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.743036420 (8h59m11.332381407s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.811114753 (8h59m11.40045974s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.889740355 (8h59m11.479085342s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996286117 (8h59m11.585631104s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996311105 (8h59m11.585656092s) [seg_pos={0 0}, upload=9375]: Segment reached optimal threshold of 14 pieces. Aug 7 18:39:11.996413861 (8h59m11.585758848s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ex8eUvL1Hqxd5EffckSvBeZDpGrMGhKENyCPvqHybtQf3YQzM): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996588316 (8h59m11.585933303s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vucBTonz7hjJgo2Jy2rhpSryqK6ifrBtj34pAcqc8FA5nPyYp): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996656285 (8h59m11.586001272s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123rwT3AmQR5nrvjoo6fgKBBJaKVWJHkJ6FtLnoNYs2AxakoDZC): protocol: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection Aug 7 18:39:11.996741468 (8h59m11.586086455s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ): protocol: read tcp 188.166.225.170:50042->104.36.82.58:4265: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:39:11.996761066 (8h59m11.586106053s) [seg_pos={0 0}, upload=9375]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:39:11.996823245 (8h59m11.586168232s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:39:11.996827292 (8h59m11.586172269s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:12.298790633 (8h59m11.88813562s) [upload=9375]: Done uploading segments... Aug 7 18:39:12.298809429 (8h59m11.888154416s) [upload=9375]: Upload finished. err: <nil> Duration: 5.769449479s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [..X..................X..X.............X......X..............] duration: 2.043871873s FAIL (2025-08-07 18:51:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 2.043899597s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................X................X...............] duration: 5.857094035s FAIL (2025-08-07 18:51:23 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== m8.735137734s) [upload=9375]: Uploading segments... Aug 7 18:39:09.145944347 (8h59m8.735289335s) [upload=9375]: Got next segment. Inline: false Aug 7 18:39:09.145982490 (8h59m8.735327467s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:09.145986848 (8h59m8.735331835s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:39:09.642154224 (8h59m9.231499201s) [seg_pos={0 0}, upload=9375]: Begin upload segment... Aug 7 18:39:09.642193350 (8h59m9.231538327s) [seg_pos={0 0}, upload=9375]: Done begin upload segment. Aug 7 18:39:09.642214950 (8h59m9.231559937s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642229899 (8h59m9.231574896s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642582825 (8h59m9.231927812s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642865325 (8h59m9.232210302s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643156402 (8h59m9.232501389s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643442810 (8h59m9.232787787s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643676567 (8h59m9.233021554s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643970980 (8h59m9.233315967s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644264642 (8h59m9.23360963s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644537404 (8h59m9.233882392s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644749891 (8h59m9.234094868s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644963820 (8h59m9.234308807s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645264726 (8h59m9.234609713s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645483795 (8h59m9.234828782s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645575301 (8h59m9.234920288s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645847200 (8h59m9.235192177s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646083532 (8h59m9.235428519s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646308663 (8h59m9.23565365s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:10.758983522 (8h59m10.348328509s) [upload=9375]: Next returned nil segment Aug 7 18:39:10.758998891 (8h59m10.348343878s) [upload=9375]: Waiting for error group managing segments... Aug 7 18:39:11.040903333 (8h59m10.63024832s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.575207667 (8h59m11.164552654s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.580126699 (8h59m11.169471696s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.620341280 (8h59m11.209686277s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.641699104 (8h59m11.231044091s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.647426883 (8h59m11.23677187s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.662592328 (8h59m11.251937305s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.697065164 (8h59m11.286410151s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.720412004 (8h59m11.309756991s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.731711513 (8h59m11.3210565s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.743036420 (8h59m11.332381407s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.811114753 (8h59m11.40045974s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.889740355 (8h59m11.479085342s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996286117 (8h59m11.585631104s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996311105 (8h59m11.585656092s) [seg_pos={0 0}, upload=9375]: Segment reached optimal threshold of 14 pieces. Aug 7 18:39:11.996413861 (8h59m11.585758848s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ex8eUvL1Hqxd5EffckSvBeZDpGrMGhKENyCPvqHybtQf3YQzM): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996588316 (8h59m11.585933303s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vucBTonz7hjJgo2Jy2rhpSryqK6ifrBtj34pAcqc8FA5nPyYp): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996656285 (8h59m11.586001272s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123rwT3AmQR5nrvjoo6fgKBBJaKVWJHkJ6FtLnoNYs2AxakoDZC): protocol: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection Aug 7 18:39:11.996741468 (8h59m11.586086455s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ): protocol: read tcp 188.166.225.170:50042->104.36.82.58:4265: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:39:11.996761066 (8h59m11.586106053s) [seg_pos={0 0}, upload=9375]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:39:11.996823245 (8h59m11.586168232s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:39:11.996827292 (8h59m11.586172269s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:12.298790633 (8h59m11.88813562s) [upload=9375]: Done uploading segments... Aug 7 18:39:12.298809429 (8h59m11.888154416s) [upload=9375]: Upload finished. err: <nil> Duration: 5.857124606s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [...X..................X..X.............X......X.............] duration: 1.860861812s FAIL (2025-08-07 18:51:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.860901607s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................X................X..............] duration: 6.079968346s FAIL (2025-08-07 18:51:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== m8.735137734s) [upload=9375]: Uploading segments... Aug 7 18:39:09.145944347 (8h59m8.735289335s) [upload=9375]: Got next segment. Inline: false Aug 7 18:39:09.145982490 (8h59m8.735327467s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:09.145986848 (8h59m8.735331835s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:39:09.642154224 (8h59m9.231499201s) [seg_pos={0 0}, upload=9375]: Begin upload segment... Aug 7 18:39:09.642193350 (8h59m9.231538327s) [seg_pos={0 0}, upload=9375]: Done begin upload segment. Aug 7 18:39:09.642214950 (8h59m9.231559937s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642229899 (8h59m9.231574896s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642582825 (8h59m9.231927812s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642865325 (8h59m9.232210302s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643156402 (8h59m9.232501389s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643442810 (8h59m9.232787787s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643676567 (8h59m9.233021554s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643970980 (8h59m9.233315967s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644264642 (8h59m9.23360963s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644537404 (8h59m9.233882392s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644749891 (8h59m9.234094868s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644963820 (8h59m9.234308807s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645264726 (8h59m9.234609713s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645483795 (8h59m9.234828782s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645575301 (8h59m9.234920288s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645847200 (8h59m9.235192177s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646083532 (8h59m9.235428519s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646308663 (8h59m9.23565365s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:10.758983522 (8h59m10.348328509s) [upload=9375]: Next returned nil segment Aug 7 18:39:10.758998891 (8h59m10.348343878s) [upload=9375]: Waiting for error group managing segments... Aug 7 18:39:11.040903333 (8h59m10.63024832s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.575207667 (8h59m11.164552654s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.580126699 (8h59m11.169471696s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.620341280 (8h59m11.209686277s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.641699104 (8h59m11.231044091s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.647426883 (8h59m11.23677187s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.662592328 (8h59m11.251937305s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.697065164 (8h59m11.286410151s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.720412004 (8h59m11.309756991s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.731711513 (8h59m11.3210565s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.743036420 (8h59m11.332381407s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.811114753 (8h59m11.40045974s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.889740355 (8h59m11.479085342s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996286117 (8h59m11.585631104s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996311105 (8h59m11.585656092s) [seg_pos={0 0}, upload=9375]: Segment reached optimal threshold of 14 pieces. Aug 7 18:39:11.996413861 (8h59m11.585758848s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ex8eUvL1Hqxd5EffckSvBeZDpGrMGhKENyCPvqHybtQf3YQzM): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996588316 (8h59m11.585933303s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vucBTonz7hjJgo2Jy2rhpSryqK6ifrBtj34pAcqc8FA5nPyYp): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996656285 (8h59m11.586001272s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123rwT3AmQR5nrvjoo6fgKBBJaKVWJHkJ6FtLnoNYs2AxakoDZC): protocol: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection Aug 7 18:39:11.996741468 (8h59m11.586086455s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ): protocol: read tcp 188.166.225.170:50042->104.36.82.58:4265: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:39:11.996761066 (8h59m11.586106053s) [seg_pos={0 0}, upload=9375]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:39:11.996823245 (8h59m11.586168232s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:39:11.996827292 (8h59m11.586172269s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:12.298790633 (8h59m11.88813562s) [upload=9375]: Done uploading segments... Aug 7 18:39:12.298809429 (8h59m11.888154416s) [upload=9375]: Upload finished. err: <nil> Duration: 6.079998082s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [....X..................X..X.............X......X............] duration: 1.86041721s FAIL (2025-08-07 18:50:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.86044374s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................X................X.............] duration: 5.568107763s FAIL (2025-08-07 18:50:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== m8.735137734s) [upload=9375]: Uploading segments... Aug 7 18:39:09.145944347 (8h59m8.735289335s) [upload=9375]: Got next segment. Inline: false Aug 7 18:39:09.145982490 (8h59m8.735327467s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:09.145986848 (8h59m8.735331835s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:39:09.642154224 (8h59m9.231499201s) [seg_pos={0 0}, upload=9375]: Begin upload segment... Aug 7 18:39:09.642193350 (8h59m9.231538327s) [seg_pos={0 0}, upload=9375]: Done begin upload segment. Aug 7 18:39:09.642214950 (8h59m9.231559937s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642229899 (8h59m9.231574896s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642582825 (8h59m9.231927812s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642865325 (8h59m9.232210302s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643156402 (8h59m9.232501389s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643442810 (8h59m9.232787787s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643676567 (8h59m9.233021554s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643970980 (8h59m9.233315967s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644264642 (8h59m9.23360963s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644537404 (8h59m9.233882392s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644749891 (8h59m9.234094868s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644963820 (8h59m9.234308807s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645264726 (8h59m9.234609713s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645483795 (8h59m9.234828782s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645575301 (8h59m9.234920288s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645847200 (8h59m9.235192177s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646083532 (8h59m9.235428519s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646308663 (8h59m9.23565365s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:10.758983522 (8h59m10.348328509s) [upload=9375]: Next returned nil segment Aug 7 18:39:10.758998891 (8h59m10.348343878s) [upload=9375]: Waiting for error group managing segments... Aug 7 18:39:11.040903333 (8h59m10.63024832s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.575207667 (8h59m11.164552654s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.580126699 (8h59m11.169471696s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.620341280 (8h59m11.209686277s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.641699104 (8h59m11.231044091s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.647426883 (8h59m11.23677187s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.662592328 (8h59m11.251937305s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.697065164 (8h59m11.286410151s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.720412004 (8h59m11.309756991s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.731711513 (8h59m11.3210565s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.743036420 (8h59m11.332381407s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.811114753 (8h59m11.40045974s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.889740355 (8h59m11.479085342s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996286117 (8h59m11.585631104s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996311105 (8h59m11.585656092s) [seg_pos={0 0}, upload=9375]: Segment reached optimal threshold of 14 pieces. Aug 7 18:39:11.996413861 (8h59m11.585758848s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ex8eUvL1Hqxd5EffckSvBeZDpGrMGhKENyCPvqHybtQf3YQzM): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996588316 (8h59m11.585933303s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vucBTonz7hjJgo2Jy2rhpSryqK6ifrBtj34pAcqc8FA5nPyYp): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996656285 (8h59m11.586001272s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123rwT3AmQR5nrvjoo6fgKBBJaKVWJHkJ6FtLnoNYs2AxakoDZC): protocol: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection Aug 7 18:39:11.996741468 (8h59m11.586086455s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ): protocol: read tcp 188.166.225.170:50042->104.36.82.58:4265: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:39:11.996761066 (8h59m11.586106053s) [seg_pos={0 0}, upload=9375]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:39:11.996823245 (8h59m11.586168232s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:39:11.996827292 (8h59m11.586172269s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:12.298790633 (8h59m11.88813562s) [upload=9375]: Done uploading segments... Aug 7 18:39:12.298809429 (8h59m11.888154416s) [upload=9375]: Upload finished. err: <nil> Duration: 5.568132772s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [X....X..................X..X.............X......X...........] duration: 1.883416323s FAIL (2025-08-07 18:49:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.883440168s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................X................X............] duration: 5.970935751s FAIL (2025-08-07 18:49:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== m8.735137734s) [upload=9375]: Uploading segments... Aug 7 18:39:09.145944347 (8h59m8.735289335s) [upload=9375]: Got next segment. Inline: false Aug 7 18:39:09.145982490 (8h59m8.735327467s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:09.145986848 (8h59m8.735331835s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 18:39:09.642154224 (8h59m9.231499201s) [seg_pos={0 0}, upload=9375]: Begin upload segment... Aug 7 18:39:09.642193350 (8h59m9.231538327s) [seg_pos={0 0}, upload=9375]: Done begin upload segment. Aug 7 18:39:09.642214950 (8h59m9.231559937s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642229899 (8h59m9.231574896s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642582825 (8h59m9.231927812s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.642865325 (8h59m9.232210302s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643156402 (8h59m9.232501389s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643442810 (8h59m9.232787787s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643676567 (8h59m9.233021554s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.643970980 (8h59m9.233315967s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644264642 (8h59m9.23360963s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644537404 (8h59m9.233882392s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644749891 (8h59m9.234094868s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.644963820 (8h59m9.234308807s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645264726 (8h59m9.234609713s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645483795 (8h59m9.234828782s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645575301 (8h59m9.234920288s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.645847200 (8h59m9.235192177s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646083532 (8h59m9.235428519s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:09.646308663 (8h59m9.23565365s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Uploading piece... Aug 7 18:39:10.758983522 (8h59m10.348328509s) [upload=9375]: Next returned nil segment Aug 7 18:39:10.758998891 (8h59m10.348343878s) [upload=9375]: Waiting for error group managing segments... Aug 7 18:39:11.040903333 (8h59m10.63024832s) [noise=true , address=64.49.13.169:4708 , piece_id=GU5AR5B6OBHOUEKMSIINQEWOPZE6YWKNT5YLQVQHFAAVZPSMTJJQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.575207667 (8h59m11.164552654s) [noise=true , address=104.36.82.54:4178 , piece_id=Q2JXOKOS4GIPCKW4XFYVC5IUNWSSMYVEMYPA4V32QF6R7ZF7LI4A, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.580126699 (8h59m11.169471696s) [noise=true , address=104.36.82.16:4313 , piece_id=RKEGZWGLQSM2MOVSQSLKDF5MWYWXFKZW23L4D546523J7NT4H7RA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.620341280 (8h59m11.209686277s) [noise=true , address=45.91.116.9:4271 , piece_id=TWEGSTML3OCBXH6KMSTBAWDWDJZUOJW367CI3BZ4IWDSNNEJB3GQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.641699104 (8h59m11.231044091s) [noise=true , address=104.36.82.14:4227 , piece_id=WN4X63JK4I2RSPQFTJZLNWY2NFMXCDNIMHI7DSC7SUHWITXPYOXA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.647426883 (8h59m11.23677187s) [noise=true , address=104.36.82.15:4292 , piece_id=TEN44N2ADRM26IVTIDLK23AAV2VGHP4YDQJTRNXU53UJFOYXPP2Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.662592328 (8h59m11.251937305s) [noise=true , address=45.91.116.7:4036 , piece_id=6XWFF4RC7WVWXSOBSMPNHKZ6LSUKQX2Z56EYJ3ABJRFCQPP3CMBQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.697065164 (8h59m11.286410151s) [noise=true , address=104.36.82.7:4940 , piece_id=EQCL6CLNB5DT43KFCJ5TI4YQFFC6RXHV3V7YB2K7JZL7YCFIUZFA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.720412004 (8h59m11.309756991s) [noise=true , address=104.36.82.41:4767 , piece_id=3ZQVUY5A36RWLZCNPZCZ5MOUV3QO5GTYCMZA5WD4GKU4S6S3WFAA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.731711513 (8h59m11.3210565s) [noise=true , address=104.36.82.52:4092 , piece_id=MHL7E5ENKVOUYXBBEKGYTQOVLCNBO6XHQXN4Q5UNBUMHWHLOQAPA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.743036420 (8h59m11.332381407s) [noise=true , address=104.36.82.36:4647 , piece_id=VF3SOIIQBVSJWIUOBYEWIEPKMHSDB2FZGHUUN2433GWK2RGE42KA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.811114753 (8h59m11.40045974s) [noise=true , address=104.36.82.61:4403 , piece_id=MQBI6SHX7F5I3ZTFUJYL4G2NQEE4OGNGNEOFBKRBDZWKMH6Z2A4Q, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.889740355 (8h59m11.479085342s) [noise=true , address=104.36.82.8:4004 , piece_id=SCTSWAN4URCI65DMWWFA3LTOSW5QJD2PAPJSANHQMJAMOLGTPGRA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996286117 (8h59m11.585631104s) [noise=true , address=104.36.82.47:4534 , piece_id=IWFNLC57APDG37Y6M25RGVTD6P2JSQOKMHJHOQNHMPGZU4GG4ASA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: <nil> Aug 7 18:39:11.996311105 (8h59m11.585656092s) [seg_pos={0 0}, upload=9375]: Segment reached optimal threshold of 14 pieces. Aug 7 18:39:11.996413861 (8h59m11.585758848s) [noise=true , address=104.36.82.35:4607 , piece_id=4VAVB44XR5XGKU6Q4EEUV5XHSCMBDQJZXPOSKSWQCAQ4HAPTEBKA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Ex8eUvL1Hqxd5EffckSvBeZDpGrMGhKENyCPvqHybtQf3YQzM): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996588316 (8h59m11.585933303s) [noise=true , address=104.36.82.9:4035 , piece_id=ASTE7IA5YGHIKR5SNNDP255OQI73Q6HBWU5MKV3WMMSR55FSINGQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vucBTonz7hjJgo2Jy2rhpSryqK6ifrBtj34pAcqc8FA5nPyYp): protocol: expected piece hash; context canceled; EOF Aug 7 18:39:11.996656285 (8h59m11.586001272s) [noise=true , address=104.36.82.37:4684 , piece_id=HG6HB7M3EQOMROV2MCDKK52NPYP6ANX2NI54N4P34ZOLCXTOEWZA, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123rwT3AmQR5nrvjoo6fgKBBJaKVWJHkJ6FtLnoNYs2AxakoDZC): protocol: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46234->104.36.82.37:4684: use of closed network connection Aug 7 18:39:11.996741468 (8h59m11.586086455s) [noise=true , address=104.36.82.58:4265 , piece_id=OTBURQWO3CQ6ZR34CN5B3JE7BI3SKMDT4OBAV6FAHAYJWIRAKTDQ, seg_pos={0 0}, upload=9375]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12JvnJetuaEkstkPVNChKo8YQEYC68qRXPGQxx2AKZoHftkhajQ): protocol: read tcp 188.166.225.170:50042->104.36.82.58:4265: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 18:39:11.996761066 (8h59m11.586106053s) [seg_pos={0 0}, upload=9375]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 18:39:11.996823245 (8h59m11.586168232s) [upload=9375]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 18:39:11.996827292 (8h59m11.586172269s) [upload=9375]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:12.298790633 (8h59m11.88813562s) [upload=9375]: Done uploading segments... Aug 7 18:39:12.298809429 (8h59m11.888154416s) [upload=9375]: Upload finished. err: <nil> Duration: 5.970970777s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [.X....X..................X..X.............X......X..........] duration: 1.86080428s FAIL (2025-08-07 18:49:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.860842464s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [..X....X..................X..X.............X......X.........] duration: 1.85580191s FAIL (2025-08-07 18:49:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== ad=9317]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:35:45.969768758 (8h55m45.559113735s) [upload=9317]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:35:46.242457297 (8h55m45.831802274s) [upload=9317]: Done uploading segments... Aug 7 18:35:46.242474340 (8h55m45.831819327s) [upload=9317]: Upload finished. err: <nil> Aug 7 18:36:15.033035023 (8h56m14.62238s) [upload=9325]: Starting upload Aug 7 18:36:15.033100007 (8h56m14.622444984s) [upload=9325]: Done starting upload Aug 7 18:36:15.033177425 (8h56m14.622522412s) [upload=9325]: Uploading segments... Aug 7 18:36:15.033229886 (8h56m14.622574863s) [upload=9325]: Got next segment. Inline: true Aug 7 18:36:15.033233613 (8h56m14.62257859s) [upload=9325]: Waiting for error group managing segments... Aug 7 18:36:15.033284991 (8h56m14.622629978s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:36:15.033292365 (8h56m14.622637342s) [upload=9325]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:36:15.033294900 (8h56m14.622639887s) [upload=9325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:36:15.539208023 (8h56m15.128553s) [upload=9325]: Done uploading segments... Aug 7 18:36:15.539224734 (8h56m15.128569722s) [upload=9325]: Upload finished. err: <nil> Aug 7 18:37:12.735607976 (8h57m12.324952953s) [upload=9342]: Starting upload Aug 7 18:37:12.735656910 (8h57m12.325001897s) [upload=9342]: Done starting upload Aug 7 18:37:12.735701074 (8h57m12.325046052s) [upload=9342]: Uploading segments... Aug 7 18:37:12.735729519 (8h57m12.325074506s) [upload=9342]: Got next segment. Inline: true Aug 7 18:37:12.735733116 (8h57m12.325078103s) [upload=9342]: Waiting for error group managing segments... Aug 7 18:37:12.735763684 (8h57m12.325108671s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:12.735767291 (8h57m12.325112278s) [upload=9342]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:12.735769625 (8h57m12.325114602s) [upload=9342]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:13.017846818 (8h57m12.607191805s) [upload=9342]: Done uploading segments... Aug 7 18:37:13.017882987 (8h57m12.607227964s) [upload=9342]: Upload finished. err: <nil> Aug 7 18:37:59.115964735 (8h57m58.705309722s) [upload=9356]: Starting upload Aug 7 18:37:59.116011114 (8h57m58.705356101s) [upload=9356]: Done starting upload Aug 7 18:37:59.116063825 (8h57m58.705408812s) [upload=9356]: Uploading segments... Aug 7 18:37:59.116097589 (8h57m58.705442576s) [upload=9356]: Got next segment. Inline: true Aug 7 18:37:59.116101336 (8h57m58.705446323s) [upload=9356]: Waiting for error group managing segments... Aug 7 18:37:59.116135491 (8h57m58.705480478s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:37:59.116139559 (8h57m58.705484546s) [upload=9356]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:37:59.116141774 (8h57m58.705486761s) [upload=9356]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:37:59.543610722 (8h57m59.132955709s) [upload=9356]: Done uploading segments... Aug 7 18:37:59.543626952 (8h57m59.132971939s) [upload=9356]: Upload finished. err: <nil> Aug 7 18:38:59.766551609 (8h58m59.355896596s) [upload=9373]: Starting upload Aug 7 18:38:59.766608668 (8h58m59.355953655s) [upload=9373]: Done starting upload Aug 7 18:38:59.766659726 (8h58m59.356004713s) [upload=9373]: Uploading segments... Aug 7 18:38:59.766706556 (8h58m59.356051543s) [upload=9373]: Got next segment. Inline: true Aug 7 18:38:59.766710273 (8h58m59.35605525s) [upload=9373]: Waiting for error group managing segments... Aug 7 18:38:59.766773934 (8h58m59.356118911s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:38:59.766777982 (8h58m59.356122959s) [upload=9373]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:38:59.766780206 (8h58m59.356125183s) [upload=9373]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:00.055071208 (8h58m59.644416205s) [upload=9373]: Done uploading segments... Aug 7 18:39:00.055094613 (8h58m59.64443962s) [upload=9373]: Upload finished. err: <nil> Aug 7 18:39:24.704104457 (8h59m24.293449434s) [upload=9381]: Starting upload Aug 7 18:39:24.704153941 (8h59m24.293498928s) [upload=9381]: Done starting upload Aug 7 18:39:24.704214438 (8h59m24.293559425s) [upload=9381]: Uploading segments... Aug 7 18:39:24.704258692 (8h59m24.293603669s) [upload=9381]: Got next segment. Inline: true Aug 7 18:39:24.704262038 (8h59m24.293607015s) [upload=9381]: Waiting for error group managing segments... Aug 7 18:39:24.704302626 (8h59m24.293647603s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:39:24.704307425 (8h59m24.293652402s) [upload=9381]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:39:24.704309920 (8h59m24.293654907s) [upload=9381]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:39:24.982057969 (8h59m24.571402956s) [upload=9381]: Done uploading segments... Aug 7 18:39:24.982076565 (8h59m24.571421542s) [upload=9381]: Upload finished. err: <nil> Aug 7 18:40:04.941588222 (9h0m4.530933209s) [upload=9393]: Starting upload Aug 7 18:40:04.941630744 (9h0m4.530975731s) [upload=9393]: Done starting upload Aug 7 18:40:04.941680949 (9h0m4.531025936s) [upload=9393]: Uploading segments... Aug 7 18:40:04.941728210 (9h0m4.531073197s) [upload=9393]: Got next segment. Inline: true Aug 7 18:40:04.941732237 (9h0m4.531077225s) [upload=9393]: Waiting for error group managing segments... Aug 7 18:40:04.941780089 (9h0m4.531125076s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:04.941784247 (9h0m4.531129234s) [upload=9393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:04.941786521 (9h0m4.531131508s) [upload=9393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:05.341000770 (9h0m4.930345777s) [upload=9393]: Done uploading segments... Aug 7 18:40:05.341014065 (9h0m4.930359082s) [upload=9393]: Upload finished. err: <nil> Aug 7 18:40:28.772148788 (9h0m28.361493775s) [upload=9400]: Starting upload Aug 7 18:40:28.772208712 (9h0m28.361553699s) [upload=9400]: Done starting upload Aug 7 18:40:28.772279418 (9h0m28.361624405s) [upload=9400]: Uploading segments... Aug 7 18:40:28.772312791 (9h0m28.361657768s) [upload=9400]: Got next segment. Inline: true Aug 7 18:40:28.772316458 (9h0m28.361661435s) [upload=9400]: Waiting for error group managing segments... Aug 7 18:40:28.772348840 (9h0m28.361693817s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:40:28.772352898 (9h0m28.361697885s) [upload=9400]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:40:28.772355322 (9h0m28.361700309s) [upload=9400]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:40:29.054132330 (9h0m28.643477317s) [upload=9400]: Done uploading segments... Aug 7 18:40:29.054150305 (9h0m28.643495292s) [upload=9400]: Upload finished. err: <nil> Aug 7 18:41:08.878426361 (9h1m8.467771348s) [upload=9412]: Starting upload Aug 7 18:41:08.878475905 (9h1m8.467820892s) [upload=9412]: Done starting upload Aug 7 18:41:08.878531873 (9h1m8.46787686s) [upload=9412]: Uploading segments... Aug 7 18:41:08.878565978 (9h1m8.467910965s) [upload=9412]: Got next segment. Inline: true Aug 7 18:41:08.878569975 (9h1m8.467914962s) [upload=9412]: Waiting for error group managing segments... Aug 7 18:41:08.878601926 (9h1m8.467946913s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:08.878606384 (9h1m8.467951362s) [upload=9412]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:08.878608689 (9h1m8.467953676s) [upload=9412]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:09.179199913 (9h1m8.76854489s) [upload=9412]: Done uploading segments... Aug 7 18:41:09.179216936 (9h1m8.768561923s) [upload=9412]: Upload finished. err: <nil> Aug 7 18:41:30.417411250 (9h1m30.006756237s) [upload=9416]: Starting upload Aug 7 18:41:30.417488778 (9h1m30.006833825s) [upload=9416]: Done starting upload Aug 7 18:41:30.417566065 (9h1m30.006911052s) [upload=9416]: Uploading segments... Aug 7 18:41:30.417613827 (9h1m30.006958814s) [upload=9416]: Got next segment. Inline: true Aug 7 18:41:30.417619878 (9h1m30.006964926s) [upload=9416]: Waiting for error group managing segments... Aug 7 18:41:30.417689752 (9h1m30.00703474s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:41:30.417697036 (9h1m30.007042083s) [upload=9416]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:41:30.417701645 (9h1m30.007046672s) [upload=9416]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:41:30.694976450 (9h1m30.284321437s) [upload=9416]: Done uploading segments... Aug 7 18:41:30.694994635 (9h1m30.284339622s) [upload=9416]: Upload finished. err: <nil> Aug 7 18:42:39.714361631 (9h2m39.303706618s) [upload=9435]: Starting upload Aug 7 18:42:39.714409533 (9h2m39.30375452s) [upload=9435]: Done starting upload Aug 7 18:42:39.714462384 (9h2m39.303807361s) [upload=9435]: Uploading segments... Aug 7 18:42:39.714510907 (9h2m39.303855904s) [upload=9435]: Got next segment. Inline: true Aug 7 18:42:39.714514424 (9h2m39.303859411s) [upload=9435]: Waiting for error group managing segments... Aug 7 18:42:39.714545824 (9h2m39.303890811s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 18:42:39.714549330 (9h2m39.303894317s) [upload=9435]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 18:42:39.714551615 (9h2m39.303896602s) [upload=9435]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 18:42:39.925979987 (9h2m39.515324974s) [upload=9435]: Done uploading segments... Aug 7 18:42:39.925997079 (9h2m39.515342066s) [upload=9435]: Upload finished. err: metaclient: Too Many Requests Duration: 1.855830696s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)