Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X............X.................X.....................] duration: 5.978562864s FAIL (2025-08-07 17:08:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 5.978587681s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 1.835495304s FAIL (2025-08-07 17:07:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.835519491s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X............X.................X....................] duration: 5.846461324s FAIL (2025-08-07 17:07:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 5.846485953s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 1.983585395s FAIL (2025-08-07 17:06:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.983602106s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X............X.................X...................] duration: 5.924721822s FAIL (2025-08-07 17:06:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 5.92474736s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 1.917923614s FAIL (2025-08-07 17:06:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.917948672s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X............X.................X..................] duration: 6.029027946s FAIL (2025-08-07 17:05:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 6.029053998s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.037916765s FAIL (2025-08-07 17:05:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.03794503s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 1.8378607s FAIL (2025-08-07 17:05:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.837900165s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X............X.................X.................] duration: 5.846727475s FAIL (2025-08-07 17:04:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 5.846754095s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 1.857053963s FAIL (2025-08-07 17:04:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.857075914s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 1.982837683s FAIL (2025-08-07 17:04:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.982870315s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X............X.................X................] duration: 5.956219684s FAIL (2025-08-07 17:03:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 5.956241517s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.810335939s FAIL (2025-08-07 17:03:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.810364814s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X............X.................X...............] duration: 6.061898979s FAIL (2025-08-07 17:03:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 6.061925208s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X............X.................X..............] duration: 5.678914611s FAIL (2025-08-07 17:02:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 5.67893978s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 1.850794688s FAIL (2025-08-07 17:02:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.850819707s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X............X.................X.............] duration: 6.094729502s FAIL (2025-08-07 17:01:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 6.094754381s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.846732624s FAIL (2025-08-07 17:01:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.846755759s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X............X.................X............] duration: 6.015473754s FAIL (2025-08-07 17:01:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 6.01549805s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.987566249s FAIL (2025-08-07 17:00:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.98760324s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X............X.................X...........] duration: 6.447567042s FAIL (2025-08-07 17:00:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 6.44759719s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 1.997286237s FAIL (2025-08-07 17:00:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.997324802s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X............X.................X..........] duration: 6.155498852s FAIL (2025-08-07 16:59:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 6.155521497s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.055376661s FAIL (2025-08-07 16:59:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.055414413s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X............X.................X.........] duration: 6.099388461s FAIL (2025-08-07 16:59:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 6.099417948s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 2.170457504s FAIL (2025-08-07 16:59:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.170485067s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 1.955137745s FAIL (2025-08-07 16:58:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.955161251s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.887091215s FAIL (2025-08-07 16:58:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.887111303s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X............X.................X........] duration: 6.249995546s FAIL (2025-08-07 16:58:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 6.250021957s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.82755271s FAIL (2025-08-07 16:57:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.827579822s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X............X.................X.......] duration: 5.902426507s FAIL (2025-08-07 16:57:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 5.90244946s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.853305309s FAIL (2025-08-07 16:57:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.853329375s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X............X.................X......] duration: 6.013725353s FAIL (2025-08-07 16:56:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 6.013753547s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 1.841777634s FAIL (2025-08-07 16:55:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.841801522s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X............X.................X.....] duration: 5.812017369s FAIL (2025-08-07 16:55:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 5.812039269s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.925375803s FAIL (2025-08-07 16:55:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.925401783s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X............X.................X....] duration: 5.951442318s FAIL (2025-08-07 16:54:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 5.951463819s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.999852318s FAIL (2025-08-07 16:54:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.999876465s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X............X.................X...] duration: 5.733972575s FAIL (2025-08-07 16:53:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 5.733997192s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.857034498s FAIL (2025-08-07 16:52:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.857056268s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X............X.................X..] duration: 5.823070033s FAIL (2025-08-07 16:52:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 5.823091596s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X............X.................X.] duration: 5.784043696s FAIL (2025-08-07 16:52:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 5.784076238s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.911141339s FAIL (2025-08-07 16:51:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.911172048s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X............X.................X] duration: 6.535503353s FAIL (2025-08-07 16:51:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after bc730b6270bc77179e757bc09a05e2602772135c849e5284d99f4ebc6fe46361 != before 27f3e52718ec359db3340366bd4b6d83a6d893ae3b52ea406ed7d7a76f2907cb ======LOGS====== 1:24.689035128 (7h11m24.278380115s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:51:25.153503067 (7h11m24.742848045s) [seg_pos={0 0}, upload=7479]: Begin upload segment... Aug 7 16:51:25.153541351 (7h11m24.742886338s) [seg_pos={0 0}, upload=7479]: Done begin upload segment. Aug 7 16:51:25.153567561 (7h11m24.742912548s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.153631494 (7h11m24.742976481s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154043322 (7h11m24.743388309s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154350379 (7h11m24.743695366s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154535263 (7h11m24.74388025s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154647888 (7h11m24.743992875s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154860524 (7h11m24.744205501s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.154926551 (7h11m24.744271538s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155184555 (7h11m24.744529542s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155198412 (7h11m24.744543399s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155228048 (7h11m24.744573035s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155460382 (7h11m24.744805369s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155574931 (7h11m24.744919918s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155666086 (7h11m24.745011073s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155868493 (7h11m24.74521348s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.155885817 (7h11m24.745230804s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156179177 (7h11m24.745524164s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:25.156447761 (7h11m24.745792749s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Uploading piece... Aug 7 16:51:26.266335654 (7h11m25.855680641s) [upload=7479]: Next returned nil segment Aug 7 16:51:26.266353158 (7h11m25.855698155s) [upload=7479]: Waiting for error group managing segments... Aug 7 16:51:26.572023672 (7h11m26.161368659s) [noise=true , address=64.49.13.26:4946 , piece_id=3EHHONBTSYUDWL2LWTXHYYQR5ADRKWVCABQEYJLS2STXUITLL3GQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.586183715 (7h11m26.175528702s) [noise=true , address=64.49.13.169:4714 , piece_id=MVSQ7PWNPPSW7GWG7SIOLGZ7WZV6M6LSH3YWAQGZGVHL2UBMNJSA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.597929788 (7h11m26.187274775s) [noise=true , address=79.127.205.233:4629 , piece_id=EMDNWPBRQJLW5RFCZ4CEWKBBAUPG2IBWAJVXH7RQ5FZIY5MRBRXA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:26.658242544 (7h11m26.247587531s) [noise=true , address=64.49.13.137:4501 , piece_id=JGVCBRJ5TJZWWNSSGYTNYJHMQAUB3VTGGADUR3K4IH7PQLJLRQYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.040752365 (7h11m26.630097352s) [noise=true , address=45.91.116.14:4201 , piece_id=VHLQJPYJSK4KZJ3MISMIJ5WAEFOH4SV7AOZIFXR5I3ZEPCB2ZMZA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.115954167 (7h11m26.705299155s) [noise=true , address=104.36.82.14:4244 , piece_id=F6SP3AFRYOVQ2KSABXIIRNFQTPAXDGKVG3JC2ZFD2MMP75DN5PRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.160991847 (7h11m26.750336834s) [noise=true , address=104.36.82.34:4580 , piece_id=AV4AYRAP756X7ISP5G2H4LYZU33T56LU4N7TBVK4NN7FH2NTWQKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.161660186 (7h11m26.751005173s) [noise=true , address=104.36.82.17:4337 , piece_id=3GVYFHC3M2OUSMDSHBGICZUFQLIFBJ2LOKF7QQ6CXIQRPAI2WOGA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.166204780 (7h11m26.755549778s) [noise=true , address=104.36.82.61:4376 , piece_id=D3KCMDGSRA46YOLECV4EATV4IZT7RLBPTFHS77U44HQJNWPMDJRA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.181971538 (7h11m26.771316525s) [noise=true , address=104.36.82.60:4345 , piece_id=J66GRHBZXFVH4YFWUBUC4FTISYBCZXGNEZVY3ARLUMW2LVDQ73WQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.212201917 (7h11m26.801546904s) [noise=true , address=104.36.82.10:4096 , piece_id=PGCH433S3GDNKVIOQSZBQJXIFTYZAI2LRCEK2ZALNPP42MK6M33A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.226123633 (7h11m26.81546862s) [noise=true , address=77.90.31.21:4892 , piece_id=C5R564265BEU74TKNGLB3VWCE3VDD2MLK6ZPLU2P53JJNZXTRUCQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.260468734 (7h11m26.849813721s) [noise=true , address=104.36.82.23:4478 , piece_id=PXAUEBNXWV7HFFFKCI42U3IJY3AIH3XF5IWE6RDBVXOHW73MZ76A, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.287993548 (7h11m26.877338535s) [noise=true , address=104.36.82.61:4380 , piece_id=X4WRAYWEXRFN4LHWXO7XHY5LQB2BDK6UTRK2Q4J454WIEHVLMMKA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: <nil> Aug 7 16:51:27.288019197 (7h11m26.877364184s) [seg_pos={0 0}, upload=7479]: Segment reached optimal threshold of 14 pieces. Aug 7 16:51:27.288195083 (7h11m26.87754007s) [noise=true , address=104.36.82.58:4288 , piece_id=BFOVVZE2AOW3KVAWQM3IXSMQKWVX4ZSY3ZWUXFD2UPEGURNVBFKQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuahEyfok8Hk4fBNcdrZQWoQZiCXfNWdbfqPMnbEPmdZQ3R1o): protocol: expected piece hash; context canceled; EOF Aug 7 16:51:27.288209090 (7h11m26.877554067s) [noise=true , address=104.36.82.16:4313 , piece_id=6G57SDAAHQIS6LVAJSAUHHI5M5BDMOSGVTIJXZCJ6XHZVUMSWUOA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nJzsvcbhmuGGQyUHPVALuTiwt2rmrtRkEY8R1P1AfsUr33kDA): protocol: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47296->104.36.82.16:4313: use of closed network connection Aug 7 16:51:27.288261120 (7h11m26.877606107s) [noise=true , address=104.36.82.62:4440 , piece_id=QEPA5BNNHV2372RDDT7SMMIIOR56SNREMQUDW2672RHDESB7XLEQ, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jvc3V5SqNUTwbUea883sp3jsPPu1PAzJaUwcjPjxdjm3JiF2L): protocol: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52886->104.36.82.62:4440: use of closed network connection Aug 7 16:51:27.288302378 (7h11m26.877647355s) [noise=true , address=104.36.82.47:4533 , piece_id=G4YHLAHYCMPBMNLBV2LU5AF4FHHYEEWPITDT3ZHID5OJLI35XPYA, seg_pos={0 0}, upload=7479]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JUUdAnaUPAE8tkzPvVCokFbZMwmhqZvwKHXReK9VU2QeDqavR): protocol: read tcp 188.166.225.170:49862->104.36.82.47:4533: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 16:51:27.288319842 (7h11m26.877664829s) [seg_pos={0 0}, upload=7479]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:51:27.288380969 (7h11m26.877725956s) [upload=7479]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:51:27.288384956 (7h11m26.877729943s) [upload=7479]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:51:27.615565376 (7h11m27.204910353s) [upload=7479]: Done uploading segments... Aug 7 16:51:27.615583070 (7h11m27.204928057s) [upload=7479]: Upload finished. err: <nil> Duration:* 6.535530585s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.297447582s FAIL (2025-08-07 16:51:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.297472981s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.022341356s FAIL (2025-08-07 16:50:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.022371814s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................X............X.................] duration: 5.715063106s FAIL (2025-08-07 16:50:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e begin upload segment. Aug 7 16:37:12.725588907 (6h57m12.314933884s) [noise=true , address=104.36.82.35:4614 , piece_id=AUVJDGRD26V35TEERKFA5PL5A2NZWW6TF46EXTRV3ZCHVXZXMKLQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.725616580 (6h57m12.314961567s) [noise=true , address=104.36.82.59:4318 , piece_id=TGRUBU5HFNMMWXWE45J2DWYGBZ3USWLMUNRPXJLDKK72ZF3CUI5A, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.725942442 (6h57m12.31528743s) [noise=true , address=79.127.205.233:4624 , piece_id=4AM6KEPYHBH5P2UKC2OSMLU354P3MEBNPS2COUML2U4TJJOZEHQA, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726061200 (6h57m12.315406187s) [noise=true , address=104.36.82.46:4491 , piece_id=BIKNDL3VU4FZUBORHC77QAZ7JKETRAKYWHEJSHFGCLP4P6TYM7ZQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726210135 (6h57m12.315555122s) [noise=true , address=193.149.164.143:4700 , piece_id=CWM2KI7NOT4KIQRTF5DCJJUDGDHJ5J67CVNACZY34YHSM5JLFT5Q, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726377485 (6h57m12.315722472s) [noise=true , address=104.36.82.15:4284 , piece_id=AU6Q224IRDAQAN4NMKVRBWJNT4WBNYXML346YZHKSEK5H6R5RXEA, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726444864 (6h57m12.315789851s) [noise=true , address=104.36.82.61:4375 , piece_id=HJWIGFWBJE7V7ZVBE36YOV3LV3TUYNUXXDVWOXZS5BKH4DN34JPQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726675255 (6h57m12.316020242s) [noise=true , address=104.36.82.62:4426 , piece_id=NFP6KYSHWEWMQSJAC5AFCIDMMLZVAISHC4S2XI3LLVNNNO3JVROQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726699982 (6h57m12.316044999s) [noise=true , address=104.36.82.52:4094 , piece_id=5COVVASIV62GCZTBSZFWP2NYMCQXFQEWSHMQWUKFWFK36FGKIJIQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726975209 (6h57m12.316320186s) [noise=true , address=104.36.82.37:4685 , piece_id=AMC6UMNWUTQT6MYOJKAYZLKC4NE5GE7B6AFQ6VT5FNNDFMHGM7TA, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727049531 (6h57m12.316394518s) [noise=true , address=104.36.82.23:4450 , piece_id=VDL66ISNVS6POUQPYJ762DZXOHVSUKDSP7KYB22JY6FHY6J3THYA, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727241528 (6h57m12.316586516s) [noise=true , address=104.36.82.47:4545 , piece_id=CHURFMNOQYBQ6Y6BJUKISOW6Y4IHSYXOPC5CNJYNTP5QO4LXWRMQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727265374 (6h57m12.316610361s) [noise=true , address=77.90.31.11:4569 , piece_id=5F6IZDAFNUAKF4WGPIR5O67GCMTCIHHHTF6GSQJSFVO4D6MSLREA, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727501826 (6h57m12.316846813s) [noise=true , address=104.36.82.9:4009 , piece_id=7LX4WAX77UTK3BW3LJVHP5DKUEBLKMHZYOCM2S6IQCY54YZTQM3A, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727512006 (6h57m12.316856993s) [noise=true , address=45.91.116.6:4006 , piece_id=CFGGSL4ZWDUL5VWN7R65KZR4TYFXHZ7KYVFVXUJJW2ZR62SNNWUQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727728940 (6h57m12.317073927s) [noise=true , address=79.127.201.215:4347 , piece_id=TXQ4HBPPT7HL5PVUFSLAAXJRGRW4Q7CLGRYBIBYV3QPA5JDKSXQA, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727762626 (6h57m12.317107613s) [noise=true , address=104.36.82.4:4810 , piece_id=6V6AKEFJ7QSJGKYW7DBRTCO65OKQZ2IPOP445WCU2N6R5WQ3WYMQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.729312681 (6h57m12.318657658s) [noise=true , address=64.49.13.155:4924 , piece_id=QX2TNSR4T7Z7DVD7ABTSPSCV5B6F36XWTDODXQ46AQ3VM4KRO2YQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:13.839807840 (6h57m13.429152827s) [upload=7235]: Next returned nil segment Aug 7 16:37:13.839819212 (6h57m13.429164199s) [upload=7235]: Waiting for error group managing segments... Aug 7 16:37:14.141501876 (6h57m13.730846863s) [noise=true , address=193.149.164.143:4700 , piece_id=CWM2KI7NOT4KIQRTF5DCJJUDGDHJ5J67CVNACZY34YHSM5JLFT5Q, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.154530513 (6h57m13.7438755s) [noise=true , address=79.127.205.233:4624 , piece_id=4AM6KEPYHBH5P2UKC2OSMLU354P3MEBNPS2COUML2U4TJJOZEHQA, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.189611183 (6h57m13.77895617s) [noise=true , address=64.49.13.155:4924 , piece_id=QX2TNSR4T7Z7DVD7ABTSPSCV5B6F36XWTDODXQ46AQ3VM4KRO2YQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.624400907 (6h57m14.213745894s) [noise=true , address=104.36.82.4:4810 , piece_id=6V6AKEFJ7QSJGKYW7DBRTCO65OKQZ2IPOP445WCU2N6R5WQ3WYMQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.662759559 (6h57m14.252104546s) [noise=true , address=79.127.201.215:4347 , piece_id=TXQ4HBPPT7HL5PVUFSLAAXJRGRW4Q7CLGRYBIBYV3QPA5JDKSXQA, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.679892799 (6h57m14.269237786s) [noise=true , address=104.36.82.46:4491 , piece_id=BIKNDL3VU4FZUBORHC77QAZ7JKETRAKYWHEJSHFGCLP4P6TYM7ZQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.684417537 (6h57m14.273762524s) [noise=true , address=104.36.82.37:4685 , piece_id=AMC6UMNWUTQT6MYOJKAYZLKC4NE5GE7B6AFQ6VT5FNNDFMHGM7TA, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.715443538 (6h57m14.304788525s) [noise=true , address=45.91.116.6:4006 , piece_id=CFGGSL4ZWDUL5VWN7R65KZR4TYFXHZ7KYVFVXUJJW2ZR62SNNWUQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.723422023 (6h57m14.31276701s) [noise=true , address=104.36.82.15:4284 , piece_id=AU6Q224IRDAQAN4NMKVRBWJNT4WBNYXML346YZHKSEK5H6R5RXEA, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.736696581 (6h57m14.326041568s) [noise=true , address=77.90.31.11:4569 , piece_id=5F6IZDAFNUAKF4WGPIR5O67GCMTCIHHHTF6GSQJSFVO4D6MSLREA, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.736895701 (6h57m14.326240688s) [noise=true , address=104.36.82.47:4545 , piece_id=CHURFMNOQYBQ6Y6BJUKISOW6Y4IHSYXOPC5CNJYNTP5QO4LXWRMQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.745370257 (6h57m14.334715244s) [noise=true , address=104.36.82.61:4375 , piece_id=HJWIGFWBJE7V7ZVBE36YOV3LV3TUYNUXXDVWOXZS5BKH4DN34JPQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.751324017 (6h57m14.340668994s) [noise=true , address=104.36.82.62:4426 , piece_id=NFP6KYSHWEWMQSJAC5AFCIDMMLZVAISHC4S2XI3LLVNNNO3JVROQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.751936279 (6h57m14.341281266s) [noise=true , address=104.36.82.23:4450 , piece_id=VDL66ISNVS6POUQPYJ762DZXOHVSUKDSP7KYB22JY6FHY6J3THYA, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.751956498 (6h57m14.341301475s) [seg_pos={0 0}, upload=7235]: Segment reached optimal threshold of 14 pieces. Aug 7 16:37:14.752102476 (6h57m14.341447464s) [noise=true , address=104.36.82.59:4318 , piece_id=TGRUBU5HFNMMWXWE45J2DWYGBZ3USWLMUNRPXJLDKK72ZF3CUI5A, seg_pos={0 0}, upload=7235]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12abkQbak5f8FneYvCZo5FVi4RNZZfijQZFDd4Y1aacLUNgLnyM): protocol: expected piece hash; context canceled; EOF Aug 7 16:37:14.752112156 (6h57m14.341457143s) [noise=true , address=104.36.82.35:4614 , piece_id=AUVJDGRD26V35TEERKFA5PL5A2NZWW6TF46EXTRV3ZCHVXZXMKLQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Pua5x8rAsuqYWCgoAY6fgy2SXNoDjzt7WwgMhzv1HN2KG5NY4): protocol: expected piece hash; context canceled; EOF Aug 7 16:37:14.752184413 (6h57m14.3415294s) [noise=true , address=104.36.82.9:4009 , piece_id=7LX4WAX77UTK3BW3LJVHP5DKUEBLKMHZYOCM2S6IQCY54YZTQM3A, seg_pos={0 0}, upload=7235]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12n33Xie5Hk1vaKc279HvGnL2QBEYsNzieL2grpnANJpy7Qu2wb): protocol: expected piece hash; context canceled; EOF Aug 7 16:37:14.752204362 (6h57m14.341549339s) [noise=true , address=104.36.82.52:4094 , piece_id=5COVVASIV62GCZTBSZFWP2NYMCQXFQEWSHMQWUKFWFK36FGKIJIQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QYpeobqZ5QtLcuNBh9fHhiKxnbEtpiAaagvgWyTZvGbgbFxH7): protocol: expected piece hash; context canceled; EOF Aug 7 16:37:14.752222407 (6h57m14.341567394s) [seg_pos={0 0}, upload=7235]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:37:14.752288242 (6h57m14.341633219s) [upload=7235]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:37:14.752292160 (6h57m14.341637147s) [upload=7235]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:37:15.081373424 (6h57m14.670718412s) [upload=7235]: Done uploading segments... Aug 7 16:37:15.081393744 (6h57m14.670738731s) [upload=7235]: Upload finished. err: <nil> Aug 7 16:38:15.732265245 (6h58m15.321610252s) [upload=7253]: Starting upload Aug 7 16:38:15.732359776 (6h58m15.321704814s) [upload=7253]: Done starting upload Aug 7 16:38:15.732412106 (6h58m15.321757113s) [upload=7253]: Uploading segments... Aug 7 16:38:15.732676131 (6h58m15.322021128s) [upload=7253]: Got next segment. Inline: false Aug 7 16:38:15.732706149 (6h58m15.322051146s) [upload=7253]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:38:15.733148886 (6h58m15.322493883s) [upload=7253]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:38:15.945172838 (6h58m15.534517825s) [upload=7253]: Next segment err: metaclient: Too Many Requests Aug 7 16:38:15.945196343 (6h58m15.53454133s) [upload=7253]: Done uploading segments... Aug 7 16:38:15.945212284 (6h58m15.534557271s) [upload=7253]: Upload finished. err: metaclient: Too Many Requests Duration: 5.715087123s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.029661161s FAIL (2025-08-07 16:49:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== mentMakeInline Aug 7 16:25:20.816239349 (6h45m20.405584336s) [upload=7023]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:21.096970899 (6h45m20.686315876s) [upload=7023]: Done uploading segments... Aug 7 16:25:21.096990246 (6h45m20.686335233s) [upload=7023]: Upload finished. err: <nil> Aug 7 16:25:55.056721315 (6h45m54.646066302s) [upload=7035]: Starting upload Aug 7 16:25:55.056771380 (6h45m54.646116367s) [upload=7035]: Done starting upload Aug 7 16:25:55.056819052 (6h45m54.646164039s) [upload=7035]: Uploading segments... Aug 7 16:25:55.056848137 (6h45m54.646193125s) [upload=7035]: Got next segment. Inline: true Aug 7 16:25:55.056851684 (6h45m54.646196671s) [upload=7035]: Waiting for error group managing segments... Aug 7 16:25:55.056906479 (6h45m54.646251466s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:25:55.056912321 (6h45m54.646257308s) [upload=7035]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:25:55.056914725 (6h45m54.646259712s) [upload=7035]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:25:55.562799686 (6h45m55.152144673s) [upload=7035]: Done uploading segments... Aug 7 16:25:55.562826908 (6h45m55.152171895s) [upload=7035]: Upload finished. err: <nil> Aug 7 16:26:21.858468314 (6h46m21.447813291s) [upload=7041]: Starting upload Aug 7 16:26:21.858514102 (6h46m21.447859079s) [upload=7041]: Done starting upload Aug 7 16:26:21.858592271 (6h46m21.447937258s) [upload=7041]: Uploading segments... Aug 7 16:26:21.858687593 (6h46m21.44803258s) [upload=7041]: Got next segment. Inline: true Aug 7 16:26:21.858692242 (6h46m21.448037219s) [upload=7041]: Waiting for error group managing segments... Aug 7 16:26:21.858745534 (6h46m21.448090521s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:21.858750323 (6h46m21.44809531s) [upload=7041]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:21.858752848 (6h46m21.448097835s) [upload=7041]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:22.139056659 (6h46m21.728401646s) [upload=7041]: Done uploading segments... Aug 7 16:26:22.139076296 (6h46m21.728421283s) [upload=7041]: Upload finished. err: <nil> Aug 7 16:26:57.477353003 (6h46m57.06669798s) [upload=7049]: Starting upload Aug 7 16:26:57.477397177 (6h46m57.066742154s) [upload=7049]: Done starting upload Aug 7 16:26:57.477445169 (6h46m57.066790156s) [upload=7049]: Uploading segments... Aug 7 16:26:57.477479485 (6h46m57.066824482s) [upload=7049]: Got next segment. Inline: true Aug 7 16:26:57.477483312 (6h46m57.066828309s) [upload=7049]: Waiting for error group managing segments... Aug 7 16:26:57.477522147 (6h46m57.066867134s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:26:57.477526174 (6h46m57.066871161s) [upload=7049]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:26:57.477528388 (6h46m57.066873375s) [upload=7049]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:26:57.754079640 (6h46m57.343424687s) [upload=7049]: Done uploading segments... Aug 7 16:26:57.754101803 (6h46m57.34344678s) [upload=7049]: Upload finished. err: <nil> Aug 7 16:27:12.733007348 (6h47m12.322352335s) [upload=7056]: Starting upload Aug 7 16:27:12.733051042 (6h47m12.322396029s) [upload=7056]: Done starting upload Aug 7 16:27:12.733104294 (6h47m12.322449281s) [upload=7056]: Uploading segments... Aug 7 16:27:12.733134382 (6h47m12.322479359s) [upload=7056]: Got next segment. Inline: true Aug 7 16:27:12.733138409 (6h47m12.322483396s) [upload=7056]: Waiting for error group managing segments... Aug 7 16:27:12.733172264 (6h47m12.322517251s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:12.733175871 (6h47m12.322520858s) [upload=7056]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:12.733178205 (6h47m12.322523192s) [upload=7056]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:13.002665771 (6h47m12.592010778s) [upload=7056]: Done uploading segments... Aug 7 16:27:13.002682172 (6h47m12.592027159s) [upload=7056]: Upload finished. err: <nil> Aug 7 16:27:40.565209038 (6h47m40.154554025s) [upload=7063]: Starting upload Aug 7 16:27:40.565259935 (6h47m40.154604922s) [upload=7063]: Done starting upload Aug 7 16:27:40.565315571 (6h47m40.154660558s) [upload=7063]: Uploading segments... Aug 7 16:27:40.565348976 (6h47m40.154693963s) [upload=7063]: Got next segment. Inline: true Aug 7 16:27:40.565353194 (6h47m40.154698181s) [upload=7063]: Waiting for error group managing segments... Aug 7 16:27:40.565385836 (6h47m40.154730823s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:27:40.565389623 (6h47m40.15473461s) [upload=7063]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:27:40.565391948 (6h47m40.154736925s) [upload=7063]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:27:40.849961440 (6h47m40.439306427s) [upload=7063]: Done uploading segments... Aug 7 16:27:40.849987169 (6h47m40.439332156s) [upload=7063]: Upload finished. err: <nil> Aug 7 16:28:24.356014617 (6h48m23.945359634s) [upload=7076]: Starting upload Aug 7 16:28:24.356109709 (6h48m23.945454736s) [upload=7076]: Done starting upload Aug 7 16:28:24.356203459 (6h48m23.945548476s) [upload=7076]: Uploading segments... Aug 7 16:28:24.356257973 (6h48m23.94560299s) [upload=7076]: Got next segment. Inline: true Aug 7 16:28:24.356264756 (6h48m23.945609763s) [upload=7076]: Waiting for error group managing segments... Aug 7 16:28:24.356327566 (6h48m23.945672563s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:24.356334379 (6h48m23.945679366s) [upload=7076]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:24.356339199 (6h48m23.945684196s) [upload=7076]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:24.867039141 (6h48m24.456384138s) [upload=7076]: Done uploading segments... Aug 7 16:28:24.867057687 (6h48m24.456402674s) [upload=7076]: Upload finished. err: <nil> Aug 7 16:28:50.953533517 (6h48m50.542878504s) [upload=7084]: Starting upload Aug 7 16:28:50.953581949 (6h48m50.542926926s) [upload=7084]: Done starting upload Aug 7 16:28:50.953629871 (6h48m50.542974848s) [upload=7084]: Uploading segments... Aug 7 16:28:50.953662293 (6h48m50.54300728s) [upload=7084]: Got next segment. Inline: true Aug 7 16:28:50.953666100 (6h48m50.543011087s) [upload=7084]: Waiting for error group managing segments... Aug 7 16:28:50.953702410 (6h48m50.543047397s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:28:50.953706659 (6h48m50.543051646s) [upload=7084]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:28:50.953709093 (6h48m50.54305408s) [upload=7084]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:28:51.240830436 (6h48m50.830175423s) [upload=7084]: Done uploading segments... Aug 7 16:28:51.240848812 (6h48m50.830193799s) [upload=7084]: Upload finished. err: <nil> Aug 7 16:30:04.163544016 (6h50m3.752889013s) [upload=7106]: Starting upload Aug 7 16:30:04.163610033 (6h50m3.75295505s) [upload=7106]: Done starting upload Aug 7 16:30:04.163678623 (6h50m3.753023621s) [upload=7106]: Uploading segments... Aug 7 16:30:04.163725533 (6h50m3.753070531s) [upload=7106]: Got next segment. Inline: true Aug 7 16:30:04.163734059 (6h50m3.753079047s) [upload=7106]: Waiting for error group managing segments... Aug 7 16:30:04.163785157 (6h50m3.753130144s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:30:04.163791139 (6h50m3.753136126s) [upload=7106]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:30:04.163794585 (6h50m3.753139582s) [upload=7106]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:30:04.632318313 (6h50m4.2216633s) [upload=7106]: Done uploading segments... Aug 7 16:30:04.632341728 (6h50m4.221686715s) [upload=7106]: Upload finished. err: <nil> Aug 7 16:31:13.056335618 (6h51m12.645680605s) [upload=7130]: Starting upload Aug 7 16:31:13.056393518 (6h51m12.645738505s) [upload=7130]: Done starting upload Aug 7 16:31:13.056498259 (6h51m12.645843296s) [upload=7130]: Uploading segments... Aug 7 16:31:13.056773485 (6h51m12.646118472s) [upload=7130]: Got next segment. Inline: true Aug 7 16:31:13.056783023 (6h51m12.64612801s) [upload=7130]: Waiting for error group managing segments... Aug 7 16:31:13.056844891 (6h51m12.646189878s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:31:13.056849140 (6h51m12.646194127s) [upload=7130]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:31:13.056851835 (6h51m12.646196822s) [upload=7130]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:31:13.371373984 (6h51m12.960718971s) [upload=7130]: Done uploading segments... Aug 7 16:31:13.371389734 (6h51m12.960734722s) [upload=7130]: Upload finished. err: <nil> Aug 7 16:32:04.837203281 (6h52m4.426548288s) [upload=7147]: Starting upload Aug 7 16:32:04.837279786 (6h52m4.426624793s) [upload=7147]: Done starting upload Aug 7 16:32:04.837352586 (6h52m4.426697593s) [upload=7147]: Uploading segments... Aug 7 16:32:04.837394887 (6h52m4.426739904s) [upload=7147]: Got next segment. Inline: true Aug 7 16:32:04.837400808 (6h52m4.426745815s) [upload=7147]: Waiting for error group managing segments... Aug 7 16:32:04.837452868 (6h52m4.426797885s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:32:04.837458749 (6h52m4.426803736s) [upload=7147]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 16:32:04.837462757 (6h52m4.426807754s) [upload=7147]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:32:05.048006856 (6h52m4.637351834s) [upload=7147]: Done uploading segments... Aug 7 16:32:05.048026574 (6h52m4.637371562s) [upload=7147]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.029691027s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................X............X................] duration: 6.001939185s FAIL (2025-08-07 16:49:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== e begin upload segment. Aug 7 16:37:12.725588907 (6h57m12.314933884s) [noise=true , address=104.36.82.35:4614 , piece_id=AUVJDGRD26V35TEERKFA5PL5A2NZWW6TF46EXTRV3ZCHVXZXMKLQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.725616580 (6h57m12.314961567s) [noise=true , address=104.36.82.59:4318 , piece_id=TGRUBU5HFNMMWXWE45J2DWYGBZ3USWLMUNRPXJLDKK72ZF3CUI5A, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.725942442 (6h57m12.31528743s) [noise=true , address=79.127.205.233:4624 , piece_id=4AM6KEPYHBH5P2UKC2OSMLU354P3MEBNPS2COUML2U4TJJOZEHQA, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726061200 (6h57m12.315406187s) [noise=true , address=104.36.82.46:4491 , piece_id=BIKNDL3VU4FZUBORHC77QAZ7JKETRAKYWHEJSHFGCLP4P6TYM7ZQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726210135 (6h57m12.315555122s) [noise=true , address=193.149.164.143:4700 , piece_id=CWM2KI7NOT4KIQRTF5DCJJUDGDHJ5J67CVNACZY34YHSM5JLFT5Q, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726377485 (6h57m12.315722472s) [noise=true , address=104.36.82.15:4284 , piece_id=AU6Q224IRDAQAN4NMKVRBWJNT4WBNYXML346YZHKSEK5H6R5RXEA, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726444864 (6h57m12.315789851s) [noise=true , address=104.36.82.61:4375 , piece_id=HJWIGFWBJE7V7ZVBE36YOV3LV3TUYNUXXDVWOXZS5BKH4DN34JPQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726675255 (6h57m12.316020242s) [noise=true , address=104.36.82.62:4426 , piece_id=NFP6KYSHWEWMQSJAC5AFCIDMMLZVAISHC4S2XI3LLVNNNO3JVROQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726699982 (6h57m12.316044999s) [noise=true , address=104.36.82.52:4094 , piece_id=5COVVASIV62GCZTBSZFWP2NYMCQXFQEWSHMQWUKFWFK36FGKIJIQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.726975209 (6h57m12.316320186s) [noise=true , address=104.36.82.37:4685 , piece_id=AMC6UMNWUTQT6MYOJKAYZLKC4NE5GE7B6AFQ6VT5FNNDFMHGM7TA, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727049531 (6h57m12.316394518s) [noise=true , address=104.36.82.23:4450 , piece_id=VDL66ISNVS6POUQPYJ762DZXOHVSUKDSP7KYB22JY6FHY6J3THYA, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727241528 (6h57m12.316586516s) [noise=true , address=104.36.82.47:4545 , piece_id=CHURFMNOQYBQ6Y6BJUKISOW6Y4IHSYXOPC5CNJYNTP5QO4LXWRMQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727265374 (6h57m12.316610361s) [noise=true , address=77.90.31.11:4569 , piece_id=5F6IZDAFNUAKF4WGPIR5O67GCMTCIHHHTF6GSQJSFVO4D6MSLREA, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727501826 (6h57m12.316846813s) [noise=true , address=104.36.82.9:4009 , piece_id=7LX4WAX77UTK3BW3LJVHP5DKUEBLKMHZYOCM2S6IQCY54YZTQM3A, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727512006 (6h57m12.316856993s) [noise=true , address=45.91.116.6:4006 , piece_id=CFGGSL4ZWDUL5VWN7R65KZR4TYFXHZ7KYVFVXUJJW2ZR62SNNWUQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727728940 (6h57m12.317073927s) [noise=true , address=79.127.201.215:4347 , piece_id=TXQ4HBPPT7HL5PVUFSLAAXJRGRW4Q7CLGRYBIBYV3QPA5JDKSXQA, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.727762626 (6h57m12.317107613s) [noise=true , address=104.36.82.4:4810 , piece_id=6V6AKEFJ7QSJGKYW7DBRTCO65OKQZ2IPOP445WCU2N6R5WQ3WYMQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:12.729312681 (6h57m12.318657658s) [noise=true , address=64.49.13.155:4924 , piece_id=QX2TNSR4T7Z7DVD7ABTSPSCV5B6F36XWTDODXQ46AQ3VM4KRO2YQ, seg_pos={0 0}, upload=7235]: Uploading piece... Aug 7 16:37:13.839807840 (6h57m13.429152827s) [upload=7235]: Next returned nil segment Aug 7 16:37:13.839819212 (6h57m13.429164199s) [upload=7235]: Waiting for error group managing segments... Aug 7 16:37:14.141501876 (6h57m13.730846863s) [noise=true , address=193.149.164.143:4700 , piece_id=CWM2KI7NOT4KIQRTF5DCJJUDGDHJ5J67CVNACZY34YHSM5JLFT5Q, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.154530513 (6h57m13.7438755s) [noise=true , address=79.127.205.233:4624 , piece_id=4AM6KEPYHBH5P2UKC2OSMLU354P3MEBNPS2COUML2U4TJJOZEHQA, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.189611183 (6h57m13.77895617s) [noise=true , address=64.49.13.155:4924 , piece_id=QX2TNSR4T7Z7DVD7ABTSPSCV5B6F36XWTDODXQ46AQ3VM4KRO2YQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.624400907 (6h57m14.213745894s) [noise=true , address=104.36.82.4:4810 , piece_id=6V6AKEFJ7QSJGKYW7DBRTCO65OKQZ2IPOP445WCU2N6R5WQ3WYMQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.662759559 (6h57m14.252104546s) [noise=true , address=79.127.201.215:4347 , piece_id=TXQ4HBPPT7HL5PVUFSLAAXJRGRW4Q7CLGRYBIBYV3QPA5JDKSXQA, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.679892799 (6h57m14.269237786s) [noise=true , address=104.36.82.46:4491 , piece_id=BIKNDL3VU4FZUBORHC77QAZ7JKETRAKYWHEJSHFGCLP4P6TYM7ZQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.684417537 (6h57m14.273762524s) [noise=true , address=104.36.82.37:4685 , piece_id=AMC6UMNWUTQT6MYOJKAYZLKC4NE5GE7B6AFQ6VT5FNNDFMHGM7TA, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.715443538 (6h57m14.304788525s) [noise=true , address=45.91.116.6:4006 , piece_id=CFGGSL4ZWDUL5VWN7R65KZR4TYFXHZ7KYVFVXUJJW2ZR62SNNWUQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.723422023 (6h57m14.31276701s) [noise=true , address=104.36.82.15:4284 , piece_id=AU6Q224IRDAQAN4NMKVRBWJNT4WBNYXML346YZHKSEK5H6R5RXEA, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.736696581 (6h57m14.326041568s) [noise=true , address=77.90.31.11:4569 , piece_id=5F6IZDAFNUAKF4WGPIR5O67GCMTCIHHHTF6GSQJSFVO4D6MSLREA, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.736895701 (6h57m14.326240688s) [noise=true , address=104.36.82.47:4545 , piece_id=CHURFMNOQYBQ6Y6BJUKISOW6Y4IHSYXOPC5CNJYNTP5QO4LXWRMQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.745370257 (6h57m14.334715244s) [noise=true , address=104.36.82.61:4375 , piece_id=HJWIGFWBJE7V7ZVBE36YOV3LV3TUYNUXXDVWOXZS5BKH4DN34JPQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.751324017 (6h57m14.340668994s) [noise=true , address=104.36.82.62:4426 , piece_id=NFP6KYSHWEWMQSJAC5AFCIDMMLZVAISHC4S2XI3LLVNNNO3JVROQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.751936279 (6h57m14.341281266s) [noise=true , address=104.36.82.23:4450 , piece_id=VDL66ISNVS6POUQPYJ762DZXOHVSUKDSP7KYB22JY6FHY6J3THYA, seg_pos={0 0}, upload=7235]: Done uploading piece. err: <nil> Aug 7 16:37:14.751956498 (6h57m14.341301475s) [seg_pos={0 0}, upload=7235]: Segment reached optimal threshold of 14 pieces. Aug 7 16:37:14.752102476 (6h57m14.341447464s) [noise=true , address=104.36.82.59:4318 , piece_id=TGRUBU5HFNMMWXWE45J2DWYGBZ3USWLMUNRPXJLDKK72ZF3CUI5A, seg_pos={0 0}, upload=7235]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12abkQbak5f8FneYvCZo5FVi4RNZZfijQZFDd4Y1aacLUNgLnyM): protocol: expected piece hash; context canceled; EOF Aug 7 16:37:14.752112156 (6h57m14.341457143s) [noise=true , address=104.36.82.35:4614 , piece_id=AUVJDGRD26V35TEERKFA5PL5A2NZWW6TF46EXTRV3ZCHVXZXMKLQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Pua5x8rAsuqYWCgoAY6fgy2SXNoDjzt7WwgMhzv1HN2KG5NY4): protocol: expected piece hash; context canceled; EOF Aug 7 16:37:14.752184413 (6h57m14.3415294s) [noise=true , address=104.36.82.9:4009 , piece_id=7LX4WAX77UTK3BW3LJVHP5DKUEBLKMHZYOCM2S6IQCY54YZTQM3A, seg_pos={0 0}, upload=7235]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12n33Xie5Hk1vaKc279HvGnL2QBEYsNzieL2grpnANJpy7Qu2wb): protocol: expected piece hash; context canceled; EOF Aug 7 16:37:14.752204362 (6h57m14.341549339s) [noise=true , address=104.36.82.52:4094 , piece_id=5COVVASIV62GCZTBSZFWP2NYMCQXFQEWSHMQWUKFWFK36FGKIJIQ, seg_pos={0 0}, upload=7235]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QYpeobqZ5QtLcuNBh9fHhiKxnbEtpiAaagvgWyTZvGbgbFxH7): protocol: expected piece hash; context canceled; EOF Aug 7 16:37:14.752222407 (6h57m14.341567394s) [seg_pos={0 0}, upload=7235]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 16:37:14.752288242 (6h57m14.341633219s) [upload=7235]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 16:37:14.752292160 (6h57m14.341637147s) [upload=7235]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 16:37:15.081373424 (6h57m14.670718412s) [upload=7235]: Done uploading segments... Aug 7 16:37:15.081393744 (6h57m14.670738731s) [upload=7235]: Upload finished. err: <nil> Aug 7 16:38:15.732265245 (6h58m15.321610252s) [upload=7253]: Starting upload Aug 7 16:38:15.732359776 (6h58m15.321704814s) [upload=7253]: Done starting upload Aug 7 16:38:15.732412106 (6h58m15.321757113s) [upload=7253]: Uploading segments... Aug 7 16:38:15.732676131 (6h58m15.322021128s) [upload=7253]: Got next segment. Inline: false Aug 7 16:38:15.732706149 (6h58m15.322051146s) [upload=7253]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 16:38:15.733148886 (6h58m15.322493883s) [upload=7253]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 16:38:15.945172838 (6h58m15.534517825s) [upload=7253]: Next segment err: metaclient: Too Many Requests Aug 7 16:38:15.945196343 (6h58m15.53454133s) [upload=7253]: Done uploading segments... Aug 7 16:38:15.945212284 (6h58m15.534557271s) [upload=7253]: Upload finished. err: metaclient: Too Many Requests Duration: 6.001963571s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)