Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X...................X............................X....] duration: 5.642563709s FAIL (2025-08-13 03:53:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:49:53.308968726 (106h23m47.740048857s) [upload=110735]: Starting upload Aug 13 03:49:53.309015837 (106h23m47.740095968s) [upload=110735]: Done starting upload Aug 13 03:49:53.309101450 (106h23m47.740181612s) [upload=110735]: Uploading segments... Aug 13 03:49:53.309234645 (106h23m47.740314807s) [upload=110735]: Got next segment. Inline: false Aug 13 03:49:53.309324146 (106h23m47.740404328s) [upload=110735]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:49:53.309331712 (106h23m47.740411893s) [upload=110735]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:49:53.766008247 (106h23m48.197088378s) [seg_pos={0 0}, upload=110735]: Begin upload segment... Aug 13 03:49:53.766052361 (106h23m48.197132482s) [seg_pos={0 0}, upload=110735]: Done begin upload segment. Aug 13 03:49:53.766070065 (106h23m48.197150196s) [noise=true , address=104.36.82.5:4853 , piece_id=TQF7HVLW7FY2OL5Z4CPV3OFFBMPDQUFP5A25CJW6UR7NLHBKAIBA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766080134 (106h23m48.197160255s) [noise=true , address=104.36.82.42:4804 , piece_id=EMN2BEBRBPKDXECQUD7KCGQGYN2EEXINWILWEFISLQC23OHNRB4Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766457186 (106h23m48.197537317s) [noise=true , address=104.36.82.36:4648 , piece_id=M5PJF2D2QSL6FFJMCONFLCUY7GQCLONGV5XAMKJPSRVAUXXAHIKQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766492693 (106h23m48.197572814s) [noise=true , address=104.36.82.35:4602 , piece_id=ZXQQNNRNWYUIRDK7PAGSRGS2QPB2AEFBD6GUEOFY7DFJDO6HVJDA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766771638 (106h23m48.197851769s) [noise=true , address=104.36.82.11:4148 , piece_id=NJM4JWKBUBEALTKWNO5OXT4OAC2UABQ5KHUIFTQFVHPT2H6IPX3Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.767178185 (106h23m48.198258316s) [noise=true , address=104.36.82.23:4451 , piece_id=IPJH7NABQEDR2A5AQ2VYYMBVMZI5BUHGZTF4KP6M4GTSIKQOPR3A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768376348 (106h23m48.199456479s) [noise=true , address=79.127.219.38:4036 , piece_id=5JVQOKDLXPABT3CCZBCM7DGXVZYKWMWAQHPETXPA2PSQ4UN5SLAA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768871976 (106h23m48.199952107s) [noise=true , address=64.49.13.169:4714 , piece_id=VXCS4DXR3VJCGXL4F4KZXBHN7PPJUVZITJ3HYRN6SPWIPYXSMYPA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768886523 (106h23m48.199966655s) [noise=true , address=104.36.82.59:4316 , piece_id=UTVPQPZQXEGGN3WTLXSLLG5YKINNKAW3GDYVVGIMETIUWJ722VYA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.769341765 (106h23m48.200421896s) [noise=true , address=77.90.31.18:4822 , piece_id=4ICKC4PJKD6H4CNOAS4WOMZKEFDJ6JZMRBETBZZMAM5Y3LAGQ5NA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.769732592 (106h23m48.200812724s) [noise=true , address=104.36.82.60:4353 , piece_id=45W3E35Z3NWEJY4RYMSF6XLOB2IMSYTA2LPGW37JCV3OP5YODWWA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770050361 (106h23m48.201130492s) [noise=true , address=104.36.82.23:4475 , piece_id=GK7HYW44D6DIMEYX4KCPS6JHRQTIEDW2ROEI3MHA7EI2KXM555KQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770315267 (106h23m48.201395398s) [noise=true , address=104.36.82.58:4297 , piece_id=PTHXANV2VCJABYLKRWROSZBDVZZN7TZJIMK4GXWWS5WSF6J7ZO6A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770557250 (106h23m48.201637371s) [noise=true , address=79.127.201.215:4349 , piece_id=LF3BZJW4MY36VLQGMKZ3TYC2BHWFOPHO4DVAUE5CUDV4RN2CXX7A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770700696 (106h23m48.201780837s) [noise=true , address=104.36.82.37:4682 , piece_id=ZXQFTSV24YGFLNKIMYCDVGMDPFGQZAIW6QPC2Z5WWRU4LMWP2UVA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771050183 (106h23m48.202130315s) [noise=true , address=104.36.82.62:4440 , piece_id=NNRH5KZNUVQQMMB7Q532YKHYVILFLXUSGXPHKS6Q2H4ISM6BU55Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771365317 (106h23m48.202445448s) [noise=true , address=77.90.31.12:4618 , piece_id=K2VN2PA3EG6YQL4Q4YBMBQEGC4SOSYDAF57LO3YFMYQ37RROUXOQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771616087 (106h23m48.202696218s) [noise=true , address=77.90.31.13:4643 , piece_id=YP4TPPDWZ7II2YCKOZOEQGVQMNVUZ74E27YBOKSSC6RICQWCCECA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:54.879746695 (106h23m49.310826816s) [upload=110735]: Next returned nil segment Aug 13 03:49:54.879761273 (106h23m49.310841394s) [upload=110735]: Waiting for error group managing segments... Aug 13 03:49:55.174383545 (106h23m49.605463676s) [noise=true , address=64.49.13.169:4714 , piece_id=VXCS4DXR3VJCGXL4F4KZXBHN7PPJUVZITJ3HYRN6SPWIPYXSMYPA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.726978445 (106h23m50.158058576s) [noise=true , address=104.36.82.35:4602 , piece_id=ZXQQNNRNWYUIRDK7PAGSRGS2QPB2AEFBD6GUEOFY7DFJDO6HVJDA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.730580046 (106h23m50.161660167s) [noise=true , address=77.90.31.13:4643 , piece_id=YP4TPPDWZ7II2YCKOZOEQGVQMNVUZ74E27YBOKSSC6RICQWCCECA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.733558084 (106h23m50.164638215s) [noise=true , address=104.36.82.37:4682 , piece_id=ZXQFTSV24YGFLNKIMYCDVGMDPFGQZAIW6QPC2Z5WWRU4LMWP2UVA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.755909440 (106h23m50.186989571s) [noise=true , address=104.36.82.5:4853 , piece_id=TQF7HVLW7FY2OL5Z4CPV3OFFBMPDQUFP5A25CJW6UR7NLHBKAIBA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.758609305 (106h23m50.189689436s) [noise=true , address=77.90.31.12:4618 , piece_id=K2VN2PA3EG6YQL4Q4YBMBQEGC4SOSYDAF57LO3YFMYQ37RROUXOQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.760733891 (106h23m50.191814022s) [noise=true , address=104.36.82.59:4316 , piece_id=UTVPQPZQXEGGN3WTLXSLLG5YKINNKAW3GDYVVGIMETIUWJ722VYA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.766054742 (106h23m50.197134873s) [noise=true , address=77.90.31.18:4822 , piece_id=4ICKC4PJKD6H4CNOAS4WOMZKEFDJ6JZMRBETBZZMAM5Y3LAGQ5NA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.781322423 (106h23m50.212402604s) [noise=true , address=104.36.82.42:4804 , piece_id=EMN2BEBRBPKDXECQUD7KCGQGYN2EEXINWILWEFISLQC23OHNRB4Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.803735116 (106h23m50.234815247s) [noise=true , address=104.36.82.11:4148 , piece_id=NJM4JWKBUBEALTKWNO5OXT4OAC2UABQ5KHUIFTQFVHPT2H6IPX3Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.814330439 (106h23m50.24541057s) [noise=true , address=104.36.82.58:4297 , piece_id=PTHXANV2VCJABYLKRWROSZBDVZZN7TZJIMK4GXWWS5WSF6J7ZO6A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.841839132 (106h23m50.272919253s) [noise=true , address=104.36.82.62:4440 , piece_id=NNRH5KZNUVQQMMB7Q532YKHYVILFLXUSGXPHKS6Q2H4ISM6BU55Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.860024787 (106h23m50.291104918s) [noise=true , address=104.36.82.36:4648 , piece_id=M5PJF2D2QSL6FFJMCONFLCUY7GQCLONGV5XAMKJPSRVAUXXAHIKQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.879951554 (106h23m50.311031675s) [noise=true , address=104.36.82.60:4353 , piece_id=45W3E35Z3NWEJY4RYMSF6XLOB2IMSYTA2LPGW37JCV3OP5YODWWA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.879986100 (106h23m50.311066231s) [seg_pos={0 0}, upload=110735]: Segment reached optimal threshold of 14 pieces. Aug 13 03:49:55.880105158 (106h23m50.311185289s) [noise=true , address=104.36.82.23:4475 , piece_id=GK7HYW44D6DIMEYX4KCPS6JHRQTIEDW2ROEI3MHA7EI2KXM555KQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880168159 (106h23m50.31124829s) [noise=true , address=79.127.201.215:4349 , piece_id=LF3BZJW4MY36VLQGMKZ3TYC2BHWFOPHO4DVAUE5CUDV4RN2CXX7A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880217973 (106h23m50.311298104s) [noise=true , address=104.36.82.23:4451 , piece_id=IPJH7NABQEDR2A5AQ2VYYMBVMZI5BUHGZTF4KP6M4GTSIKQOPR3A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bC4P3xBiPFkxLvavGFVbWKnbprCStL6UJgfKSA2Wd2fX7Mbfw): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880250767 (106h23m50.311330898s) [noise=true , address=79.127.219.38:4036 , piece_id=5JVQOKDLXPABT3CCZBCM7DGXVZYKWMWAQHPETXPA2PSQ4UN5SLAA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DgDXbxPwDCoCgvrRwPxtopfsU5GzqqCtmKRHRVZmVndb1wVrm): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880282638 (106h23m50.311362769s) [seg_pos={0 0}, upload=110735]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:49:55.880377539 (106h23m50.31145766s) [upload=110735]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:49:55.880382067 (106h23m50.311462198s) [upload=110735]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:49:56.200392606 (106h23m50.631472737s) [upload=110735]: Done uploading segments... Aug 13 03:49:56.200410511 (106h23m50.631490642s) [upload=110735]: Upload finished. err: <nil> Duration: 5.642591072s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 1.985967707s FAIL (2025-08-13 03:53:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.985988116s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.926207034s FAIL (2025-08-13 03:53:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.92623122s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X.....XX...........................................] duration: 7.80259033s FAIL (2025-08-13 03:52:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.802611891s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X...................X............................X...] duration: 5.698753509s FAIL (2025-08-13 03:52:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:49:53.308968726 (106h23m47.740048857s) [upload=110735]: Starting upload Aug 13 03:49:53.309015837 (106h23m47.740095968s) [upload=110735]: Done starting upload Aug 13 03:49:53.309101450 (106h23m47.740181612s) [upload=110735]: Uploading segments... Aug 13 03:49:53.309234645 (106h23m47.740314807s) [upload=110735]: Got next segment. Inline: false Aug 13 03:49:53.309324146 (106h23m47.740404328s) [upload=110735]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:49:53.309331712 (106h23m47.740411893s) [upload=110735]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:49:53.766008247 (106h23m48.197088378s) [seg_pos={0 0}, upload=110735]: Begin upload segment... Aug 13 03:49:53.766052361 (106h23m48.197132482s) [seg_pos={0 0}, upload=110735]: Done begin upload segment. Aug 13 03:49:53.766070065 (106h23m48.197150196s) [noise=true , address=104.36.82.5:4853 , piece_id=TQF7HVLW7FY2OL5Z4CPV3OFFBMPDQUFP5A25CJW6UR7NLHBKAIBA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766080134 (106h23m48.197160255s) [noise=true , address=104.36.82.42:4804 , piece_id=EMN2BEBRBPKDXECQUD7KCGQGYN2EEXINWILWEFISLQC23OHNRB4Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766457186 (106h23m48.197537317s) [noise=true , address=104.36.82.36:4648 , piece_id=M5PJF2D2QSL6FFJMCONFLCUY7GQCLONGV5XAMKJPSRVAUXXAHIKQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766492693 (106h23m48.197572814s) [noise=true , address=104.36.82.35:4602 , piece_id=ZXQQNNRNWYUIRDK7PAGSRGS2QPB2AEFBD6GUEOFY7DFJDO6HVJDA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766771638 (106h23m48.197851769s) [noise=true , address=104.36.82.11:4148 , piece_id=NJM4JWKBUBEALTKWNO5OXT4OAC2UABQ5KHUIFTQFVHPT2H6IPX3Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.767178185 (106h23m48.198258316s) [noise=true , address=104.36.82.23:4451 , piece_id=IPJH7NABQEDR2A5AQ2VYYMBVMZI5BUHGZTF4KP6M4GTSIKQOPR3A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768376348 (106h23m48.199456479s) [noise=true , address=79.127.219.38:4036 , piece_id=5JVQOKDLXPABT3CCZBCM7DGXVZYKWMWAQHPETXPA2PSQ4UN5SLAA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768871976 (106h23m48.199952107s) [noise=true , address=64.49.13.169:4714 , piece_id=VXCS4DXR3VJCGXL4F4KZXBHN7PPJUVZITJ3HYRN6SPWIPYXSMYPA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768886523 (106h23m48.199966655s) [noise=true , address=104.36.82.59:4316 , piece_id=UTVPQPZQXEGGN3WTLXSLLG5YKINNKAW3GDYVVGIMETIUWJ722VYA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.769341765 (106h23m48.200421896s) [noise=true , address=77.90.31.18:4822 , piece_id=4ICKC4PJKD6H4CNOAS4WOMZKEFDJ6JZMRBETBZZMAM5Y3LAGQ5NA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.769732592 (106h23m48.200812724s) [noise=true , address=104.36.82.60:4353 , piece_id=45W3E35Z3NWEJY4RYMSF6XLOB2IMSYTA2LPGW37JCV3OP5YODWWA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770050361 (106h23m48.201130492s) [noise=true , address=104.36.82.23:4475 , piece_id=GK7HYW44D6DIMEYX4KCPS6JHRQTIEDW2ROEI3MHA7EI2KXM555KQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770315267 (106h23m48.201395398s) [noise=true , address=104.36.82.58:4297 , piece_id=PTHXANV2VCJABYLKRWROSZBDVZZN7TZJIMK4GXWWS5WSF6J7ZO6A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770557250 (106h23m48.201637371s) [noise=true , address=79.127.201.215:4349 , piece_id=LF3BZJW4MY36VLQGMKZ3TYC2BHWFOPHO4DVAUE5CUDV4RN2CXX7A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770700696 (106h23m48.201780837s) [noise=true , address=104.36.82.37:4682 , piece_id=ZXQFTSV24YGFLNKIMYCDVGMDPFGQZAIW6QPC2Z5WWRU4LMWP2UVA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771050183 (106h23m48.202130315s) [noise=true , address=104.36.82.62:4440 , piece_id=NNRH5KZNUVQQMMB7Q532YKHYVILFLXUSGXPHKS6Q2H4ISM6BU55Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771365317 (106h23m48.202445448s) [noise=true , address=77.90.31.12:4618 , piece_id=K2VN2PA3EG6YQL4Q4YBMBQEGC4SOSYDAF57LO3YFMYQ37RROUXOQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771616087 (106h23m48.202696218s) [noise=true , address=77.90.31.13:4643 , piece_id=YP4TPPDWZ7II2YCKOZOEQGVQMNVUZ74E27YBOKSSC6RICQWCCECA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:54.879746695 (106h23m49.310826816s) [upload=110735]: Next returned nil segment Aug 13 03:49:54.879761273 (106h23m49.310841394s) [upload=110735]: Waiting for error group managing segments... Aug 13 03:49:55.174383545 (106h23m49.605463676s) [noise=true , address=64.49.13.169:4714 , piece_id=VXCS4DXR3VJCGXL4F4KZXBHN7PPJUVZITJ3HYRN6SPWIPYXSMYPA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.726978445 (106h23m50.158058576s) [noise=true , address=104.36.82.35:4602 , piece_id=ZXQQNNRNWYUIRDK7PAGSRGS2QPB2AEFBD6GUEOFY7DFJDO6HVJDA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.730580046 (106h23m50.161660167s) [noise=true , address=77.90.31.13:4643 , piece_id=YP4TPPDWZ7II2YCKOZOEQGVQMNVUZ74E27YBOKSSC6RICQWCCECA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.733558084 (106h23m50.164638215s) [noise=true , address=104.36.82.37:4682 , piece_id=ZXQFTSV24YGFLNKIMYCDVGMDPFGQZAIW6QPC2Z5WWRU4LMWP2UVA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.755909440 (106h23m50.186989571s) [noise=true , address=104.36.82.5:4853 , piece_id=TQF7HVLW7FY2OL5Z4CPV3OFFBMPDQUFP5A25CJW6UR7NLHBKAIBA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.758609305 (106h23m50.189689436s) [noise=true , address=77.90.31.12:4618 , piece_id=K2VN2PA3EG6YQL4Q4YBMBQEGC4SOSYDAF57LO3YFMYQ37RROUXOQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.760733891 (106h23m50.191814022s) [noise=true , address=104.36.82.59:4316 , piece_id=UTVPQPZQXEGGN3WTLXSLLG5YKINNKAW3GDYVVGIMETIUWJ722VYA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.766054742 (106h23m50.197134873s) [noise=true , address=77.90.31.18:4822 , piece_id=4ICKC4PJKD6H4CNOAS4WOMZKEFDJ6JZMRBETBZZMAM5Y3LAGQ5NA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.781322423 (106h23m50.212402604s) [noise=true , address=104.36.82.42:4804 , piece_id=EMN2BEBRBPKDXECQUD7KCGQGYN2EEXINWILWEFISLQC23OHNRB4Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.803735116 (106h23m50.234815247s) [noise=true , address=104.36.82.11:4148 , piece_id=NJM4JWKBUBEALTKWNO5OXT4OAC2UABQ5KHUIFTQFVHPT2H6IPX3Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.814330439 (106h23m50.24541057s) [noise=true , address=104.36.82.58:4297 , piece_id=PTHXANV2VCJABYLKRWROSZBDVZZN7TZJIMK4GXWWS5WSF6J7ZO6A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.841839132 (106h23m50.272919253s) [noise=true , address=104.36.82.62:4440 , piece_id=NNRH5KZNUVQQMMB7Q532YKHYVILFLXUSGXPHKS6Q2H4ISM6BU55Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.860024787 (106h23m50.291104918s) [noise=true , address=104.36.82.36:4648 , piece_id=M5PJF2D2QSL6FFJMCONFLCUY7GQCLONGV5XAMKJPSRVAUXXAHIKQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.879951554 (106h23m50.311031675s) [noise=true , address=104.36.82.60:4353 , piece_id=45W3E35Z3NWEJY4RYMSF6XLOB2IMSYTA2LPGW37JCV3OP5YODWWA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.879986100 (106h23m50.311066231s) [seg_pos={0 0}, upload=110735]: Segment reached optimal threshold of 14 pieces. Aug 13 03:49:55.880105158 (106h23m50.311185289s) [noise=true , address=104.36.82.23:4475 , piece_id=GK7HYW44D6DIMEYX4KCPS6JHRQTIEDW2ROEI3MHA7EI2KXM555KQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880168159 (106h23m50.31124829s) [noise=true , address=79.127.201.215:4349 , piece_id=LF3BZJW4MY36VLQGMKZ3TYC2BHWFOPHO4DVAUE5CUDV4RN2CXX7A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880217973 (106h23m50.311298104s) [noise=true , address=104.36.82.23:4451 , piece_id=IPJH7NABQEDR2A5AQ2VYYMBVMZI5BUHGZTF4KP6M4GTSIKQOPR3A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bC4P3xBiPFkxLvavGFVbWKnbprCStL6UJgfKSA2Wd2fX7Mbfw): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880250767 (106h23m50.311330898s) [noise=true , address=79.127.219.38:4036 , piece_id=5JVQOKDLXPABT3CCZBCM7DGXVZYKWMWAQHPETXPA2PSQ4UN5SLAA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DgDXbxPwDCoCgvrRwPxtopfsU5GzqqCtmKRHRVZmVndb1wVrm): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880282638 (106h23m50.311362769s) [seg_pos={0 0}, upload=110735]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:49:55.880377539 (106h23m50.31145766s) [upload=110735]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:49:55.880382067 (106h23m50.311462198s) [upload=110735]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:49:56.200392606 (106h23m50.631472737s) [upload=110735]: Done uploading segments... Aug 13 03:49:56.200410511 (106h23m50.631490642s) [upload=110735]: Upload finished. err: <nil> Duration: 5.698775082s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X.....XX..........................................] duration: 8.115063596s FAIL (2025-08-13 03:52:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.115088042s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X.....XX.........................................] duration: 7.363327373s FAIL (2025-08-13 03:52:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.363349305s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.014853966s FAIL (2025-08-13 03:52:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.014880328s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X...................X............................X..] duration: 5.658591617s FAIL (2025-08-13 03:51:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:49:53.308968726 (106h23m47.740048857s) [upload=110735]: Starting upload Aug 13 03:49:53.309015837 (106h23m47.740095968s) [upload=110735]: Done starting upload Aug 13 03:49:53.309101450 (106h23m47.740181612s) [upload=110735]: Uploading segments... Aug 13 03:49:53.309234645 (106h23m47.740314807s) [upload=110735]: Got next segment. Inline: false Aug 13 03:49:53.309324146 (106h23m47.740404328s) [upload=110735]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:49:53.309331712 (106h23m47.740411893s) [upload=110735]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:49:53.766008247 (106h23m48.197088378s) [seg_pos={0 0}, upload=110735]: Begin upload segment... Aug 13 03:49:53.766052361 (106h23m48.197132482s) [seg_pos={0 0}, upload=110735]: Done begin upload segment. Aug 13 03:49:53.766070065 (106h23m48.197150196s) [noise=true , address=104.36.82.5:4853 , piece_id=TQF7HVLW7FY2OL5Z4CPV3OFFBMPDQUFP5A25CJW6UR7NLHBKAIBA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766080134 (106h23m48.197160255s) [noise=true , address=104.36.82.42:4804 , piece_id=EMN2BEBRBPKDXECQUD7KCGQGYN2EEXINWILWEFISLQC23OHNRB4Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766457186 (106h23m48.197537317s) [noise=true , address=104.36.82.36:4648 , piece_id=M5PJF2D2QSL6FFJMCONFLCUY7GQCLONGV5XAMKJPSRVAUXXAHIKQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766492693 (106h23m48.197572814s) [noise=true , address=104.36.82.35:4602 , piece_id=ZXQQNNRNWYUIRDK7PAGSRGS2QPB2AEFBD6GUEOFY7DFJDO6HVJDA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766771638 (106h23m48.197851769s) [noise=true , address=104.36.82.11:4148 , piece_id=NJM4JWKBUBEALTKWNO5OXT4OAC2UABQ5KHUIFTQFVHPT2H6IPX3Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.767178185 (106h23m48.198258316s) [noise=true , address=104.36.82.23:4451 , piece_id=IPJH7NABQEDR2A5AQ2VYYMBVMZI5BUHGZTF4KP6M4GTSIKQOPR3A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768376348 (106h23m48.199456479s) [noise=true , address=79.127.219.38:4036 , piece_id=5JVQOKDLXPABT3CCZBCM7DGXVZYKWMWAQHPETXPA2PSQ4UN5SLAA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768871976 (106h23m48.199952107s) [noise=true , address=64.49.13.169:4714 , piece_id=VXCS4DXR3VJCGXL4F4KZXBHN7PPJUVZITJ3HYRN6SPWIPYXSMYPA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768886523 (106h23m48.199966655s) [noise=true , address=104.36.82.59:4316 , piece_id=UTVPQPZQXEGGN3WTLXSLLG5YKINNKAW3GDYVVGIMETIUWJ722VYA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.769341765 (106h23m48.200421896s) [noise=true , address=77.90.31.18:4822 , piece_id=4ICKC4PJKD6H4CNOAS4WOMZKEFDJ6JZMRBETBZZMAM5Y3LAGQ5NA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.769732592 (106h23m48.200812724s) [noise=true , address=104.36.82.60:4353 , piece_id=45W3E35Z3NWEJY4RYMSF6XLOB2IMSYTA2LPGW37JCV3OP5YODWWA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770050361 (106h23m48.201130492s) [noise=true , address=104.36.82.23:4475 , piece_id=GK7HYW44D6DIMEYX4KCPS6JHRQTIEDW2ROEI3MHA7EI2KXM555KQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770315267 (106h23m48.201395398s) [noise=true , address=104.36.82.58:4297 , piece_id=PTHXANV2VCJABYLKRWROSZBDVZZN7TZJIMK4GXWWS5WSF6J7ZO6A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770557250 (106h23m48.201637371s) [noise=true , address=79.127.201.215:4349 , piece_id=LF3BZJW4MY36VLQGMKZ3TYC2BHWFOPHO4DVAUE5CUDV4RN2CXX7A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770700696 (106h23m48.201780837s) [noise=true , address=104.36.82.37:4682 , piece_id=ZXQFTSV24YGFLNKIMYCDVGMDPFGQZAIW6QPC2Z5WWRU4LMWP2UVA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771050183 (106h23m48.202130315s) [noise=true , address=104.36.82.62:4440 , piece_id=NNRH5KZNUVQQMMB7Q532YKHYVILFLXUSGXPHKS6Q2H4ISM6BU55Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771365317 (106h23m48.202445448s) [noise=true , address=77.90.31.12:4618 , piece_id=K2VN2PA3EG6YQL4Q4YBMBQEGC4SOSYDAF57LO3YFMYQ37RROUXOQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771616087 (106h23m48.202696218s) [noise=true , address=77.90.31.13:4643 , piece_id=YP4TPPDWZ7II2YCKOZOEQGVQMNVUZ74E27YBOKSSC6RICQWCCECA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:54.879746695 (106h23m49.310826816s) [upload=110735]: Next returned nil segment Aug 13 03:49:54.879761273 (106h23m49.310841394s) [upload=110735]: Waiting for error group managing segments... Aug 13 03:49:55.174383545 (106h23m49.605463676s) [noise=true , address=64.49.13.169:4714 , piece_id=VXCS4DXR3VJCGXL4F4KZXBHN7PPJUVZITJ3HYRN6SPWIPYXSMYPA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.726978445 (106h23m50.158058576s) [noise=true , address=104.36.82.35:4602 , piece_id=ZXQQNNRNWYUIRDK7PAGSRGS2QPB2AEFBD6GUEOFY7DFJDO6HVJDA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.730580046 (106h23m50.161660167s) [noise=true , address=77.90.31.13:4643 , piece_id=YP4TPPDWZ7II2YCKOZOEQGVQMNVUZ74E27YBOKSSC6RICQWCCECA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.733558084 (106h23m50.164638215s) [noise=true , address=104.36.82.37:4682 , piece_id=ZXQFTSV24YGFLNKIMYCDVGMDPFGQZAIW6QPC2Z5WWRU4LMWP2UVA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.755909440 (106h23m50.186989571s) [noise=true , address=104.36.82.5:4853 , piece_id=TQF7HVLW7FY2OL5Z4CPV3OFFBMPDQUFP5A25CJW6UR7NLHBKAIBA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.758609305 (106h23m50.189689436s) [noise=true , address=77.90.31.12:4618 , piece_id=K2VN2PA3EG6YQL4Q4YBMBQEGC4SOSYDAF57LO3YFMYQ37RROUXOQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.760733891 (106h23m50.191814022s) [noise=true , address=104.36.82.59:4316 , piece_id=UTVPQPZQXEGGN3WTLXSLLG5YKINNKAW3GDYVVGIMETIUWJ722VYA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.766054742 (106h23m50.197134873s) [noise=true , address=77.90.31.18:4822 , piece_id=4ICKC4PJKD6H4CNOAS4WOMZKEFDJ6JZMRBETBZZMAM5Y3LAGQ5NA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.781322423 (106h23m50.212402604s) [noise=true , address=104.36.82.42:4804 , piece_id=EMN2BEBRBPKDXECQUD7KCGQGYN2EEXINWILWEFISLQC23OHNRB4Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.803735116 (106h23m50.234815247s) [noise=true , address=104.36.82.11:4148 , piece_id=NJM4JWKBUBEALTKWNO5OXT4OAC2UABQ5KHUIFTQFVHPT2H6IPX3Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.814330439 (106h23m50.24541057s) [noise=true , address=104.36.82.58:4297 , piece_id=PTHXANV2VCJABYLKRWROSZBDVZZN7TZJIMK4GXWWS5WSF6J7ZO6A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.841839132 (106h23m50.272919253s) [noise=true , address=104.36.82.62:4440 , piece_id=NNRH5KZNUVQQMMB7Q532YKHYVILFLXUSGXPHKS6Q2H4ISM6BU55Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.860024787 (106h23m50.291104918s) [noise=true , address=104.36.82.36:4648 , piece_id=M5PJF2D2QSL6FFJMCONFLCUY7GQCLONGV5XAMKJPSRVAUXXAHIKQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.879951554 (106h23m50.311031675s) [noise=true , address=104.36.82.60:4353 , piece_id=45W3E35Z3NWEJY4RYMSF6XLOB2IMSYTA2LPGW37JCV3OP5YODWWA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.879986100 (106h23m50.311066231s) [seg_pos={0 0}, upload=110735]: Segment reached optimal threshold of 14 pieces. Aug 13 03:49:55.880105158 (106h23m50.311185289s) [noise=true , address=104.36.82.23:4475 , piece_id=GK7HYW44D6DIMEYX4KCPS6JHRQTIEDW2ROEI3MHA7EI2KXM555KQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880168159 (106h23m50.31124829s) [noise=true , address=79.127.201.215:4349 , piece_id=LF3BZJW4MY36VLQGMKZ3TYC2BHWFOPHO4DVAUE5CUDV4RN2CXX7A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880217973 (106h23m50.311298104s) [noise=true , address=104.36.82.23:4451 , piece_id=IPJH7NABQEDR2A5AQ2VYYMBVMZI5BUHGZTF4KP6M4GTSIKQOPR3A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bC4P3xBiPFkxLvavGFVbWKnbprCStL6UJgfKSA2Wd2fX7Mbfw): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880250767 (106h23m50.311330898s) [noise=true , address=79.127.219.38:4036 , piece_id=5JVQOKDLXPABT3CCZBCM7DGXVZYKWMWAQHPETXPA2PSQ4UN5SLAA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DgDXbxPwDCoCgvrRwPxtopfsU5GzqqCtmKRHRVZmVndb1wVrm): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880282638 (106h23m50.311362769s) [seg_pos={0 0}, upload=110735]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:49:55.880377539 (106h23m50.31145766s) [upload=110735]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:49:55.880382067 (106h23m50.311462198s) [upload=110735]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:49:56.200392606 (106h23m50.631472737s) [upload=110735]: Done uploading segments... Aug 13 03:49:56.200410511 (106h23m50.631490642s) [upload=110735]: Upload finished. err: <nil> Duration: 5.658612458s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X.....XX........................................] duration: 7.510018062s FAIL (2025-08-13 03:51:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.510064061s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.800146547s FAIL (2025-08-13 03:51:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.800176235s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X...................X............................X.] duration: 5.67547308s FAIL (2025-08-13 03:50:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:49:53.308968726 (106h23m47.740048857s) [upload=110735]: Starting upload Aug 13 03:49:53.309015837 (106h23m47.740095968s) [upload=110735]: Done starting upload Aug 13 03:49:53.309101450 (106h23m47.740181612s) [upload=110735]: Uploading segments... Aug 13 03:49:53.309234645 (106h23m47.740314807s) [upload=110735]: Got next segment. Inline: false Aug 13 03:49:53.309324146 (106h23m47.740404328s) [upload=110735]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:49:53.309331712 (106h23m47.740411893s) [upload=110735]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:49:53.766008247 (106h23m48.197088378s) [seg_pos={0 0}, upload=110735]: Begin upload segment... Aug 13 03:49:53.766052361 (106h23m48.197132482s) [seg_pos={0 0}, upload=110735]: Done begin upload segment. Aug 13 03:49:53.766070065 (106h23m48.197150196s) [noise=true , address=104.36.82.5:4853 , piece_id=TQF7HVLW7FY2OL5Z4CPV3OFFBMPDQUFP5A25CJW6UR7NLHBKAIBA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766080134 (106h23m48.197160255s) [noise=true , address=104.36.82.42:4804 , piece_id=EMN2BEBRBPKDXECQUD7KCGQGYN2EEXINWILWEFISLQC23OHNRB4Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766457186 (106h23m48.197537317s) [noise=true , address=104.36.82.36:4648 , piece_id=M5PJF2D2QSL6FFJMCONFLCUY7GQCLONGV5XAMKJPSRVAUXXAHIKQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766492693 (106h23m48.197572814s) [noise=true , address=104.36.82.35:4602 , piece_id=ZXQQNNRNWYUIRDK7PAGSRGS2QPB2AEFBD6GUEOFY7DFJDO6HVJDA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766771638 (106h23m48.197851769s) [noise=true , address=104.36.82.11:4148 , piece_id=NJM4JWKBUBEALTKWNO5OXT4OAC2UABQ5KHUIFTQFVHPT2H6IPX3Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.767178185 (106h23m48.198258316s) [noise=true , address=104.36.82.23:4451 , piece_id=IPJH7NABQEDR2A5AQ2VYYMBVMZI5BUHGZTF4KP6M4GTSIKQOPR3A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768376348 (106h23m48.199456479s) [noise=true , address=79.127.219.38:4036 , piece_id=5JVQOKDLXPABT3CCZBCM7DGXVZYKWMWAQHPETXPA2PSQ4UN5SLAA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768871976 (106h23m48.199952107s) [noise=true , address=64.49.13.169:4714 , piece_id=VXCS4DXR3VJCGXL4F4KZXBHN7PPJUVZITJ3HYRN6SPWIPYXSMYPA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768886523 (106h23m48.199966655s) [noise=true , address=104.36.82.59:4316 , piece_id=UTVPQPZQXEGGN3WTLXSLLG5YKINNKAW3GDYVVGIMETIUWJ722VYA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.769341765 (106h23m48.200421896s) [noise=true , address=77.90.31.18:4822 , piece_id=4ICKC4PJKD6H4CNOAS4WOMZKEFDJ6JZMRBETBZZMAM5Y3LAGQ5NA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.769732592 (106h23m48.200812724s) [noise=true , address=104.36.82.60:4353 , piece_id=45W3E35Z3NWEJY4RYMSF6XLOB2IMSYTA2LPGW37JCV3OP5YODWWA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770050361 (106h23m48.201130492s) [noise=true , address=104.36.82.23:4475 , piece_id=GK7HYW44D6DIMEYX4KCPS6JHRQTIEDW2ROEI3MHA7EI2KXM555KQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770315267 (106h23m48.201395398s) [noise=true , address=104.36.82.58:4297 , piece_id=PTHXANV2VCJABYLKRWROSZBDVZZN7TZJIMK4GXWWS5WSF6J7ZO6A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770557250 (106h23m48.201637371s) [noise=true , address=79.127.201.215:4349 , piece_id=LF3BZJW4MY36VLQGMKZ3TYC2BHWFOPHO4DVAUE5CUDV4RN2CXX7A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770700696 (106h23m48.201780837s) [noise=true , address=104.36.82.37:4682 , piece_id=ZXQFTSV24YGFLNKIMYCDVGMDPFGQZAIW6QPC2Z5WWRU4LMWP2UVA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771050183 (106h23m48.202130315s) [noise=true , address=104.36.82.62:4440 , piece_id=NNRH5KZNUVQQMMB7Q532YKHYVILFLXUSGXPHKS6Q2H4ISM6BU55Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771365317 (106h23m48.202445448s) [noise=true , address=77.90.31.12:4618 , piece_id=K2VN2PA3EG6YQL4Q4YBMBQEGC4SOSYDAF57LO3YFMYQ37RROUXOQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771616087 (106h23m48.202696218s) [noise=true , address=77.90.31.13:4643 , piece_id=YP4TPPDWZ7II2YCKOZOEQGVQMNVUZ74E27YBOKSSC6RICQWCCECA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:54.879746695 (106h23m49.310826816s) [upload=110735]: Next returned nil segment Aug 13 03:49:54.879761273 (106h23m49.310841394s) [upload=110735]: Waiting for error group managing segments... Aug 13 03:49:55.174383545 (106h23m49.605463676s) [noise=true , address=64.49.13.169:4714 , piece_id=VXCS4DXR3VJCGXL4F4KZXBHN7PPJUVZITJ3HYRN6SPWIPYXSMYPA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.726978445 (106h23m50.158058576s) [noise=true , address=104.36.82.35:4602 , piece_id=ZXQQNNRNWYUIRDK7PAGSRGS2QPB2AEFBD6GUEOFY7DFJDO6HVJDA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.730580046 (106h23m50.161660167s) [noise=true , address=77.90.31.13:4643 , piece_id=YP4TPPDWZ7II2YCKOZOEQGVQMNVUZ74E27YBOKSSC6RICQWCCECA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.733558084 (106h23m50.164638215s) [noise=true , address=104.36.82.37:4682 , piece_id=ZXQFTSV24YGFLNKIMYCDVGMDPFGQZAIW6QPC2Z5WWRU4LMWP2UVA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.755909440 (106h23m50.186989571s) [noise=true , address=104.36.82.5:4853 , piece_id=TQF7HVLW7FY2OL5Z4CPV3OFFBMPDQUFP5A25CJW6UR7NLHBKAIBA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.758609305 (106h23m50.189689436s) [noise=true , address=77.90.31.12:4618 , piece_id=K2VN2PA3EG6YQL4Q4YBMBQEGC4SOSYDAF57LO3YFMYQ37RROUXOQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.760733891 (106h23m50.191814022s) [noise=true , address=104.36.82.59:4316 , piece_id=UTVPQPZQXEGGN3WTLXSLLG5YKINNKAW3GDYVVGIMETIUWJ722VYA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.766054742 (106h23m50.197134873s) [noise=true , address=77.90.31.18:4822 , piece_id=4ICKC4PJKD6H4CNOAS4WOMZKEFDJ6JZMRBETBZZMAM5Y3LAGQ5NA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.781322423 (106h23m50.212402604s) [noise=true , address=104.36.82.42:4804 , piece_id=EMN2BEBRBPKDXECQUD7KCGQGYN2EEXINWILWEFISLQC23OHNRB4Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.803735116 (106h23m50.234815247s) [noise=true , address=104.36.82.11:4148 , piece_id=NJM4JWKBUBEALTKWNO5OXT4OAC2UABQ5KHUIFTQFVHPT2H6IPX3Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.814330439 (106h23m50.24541057s) [noise=true , address=104.36.82.58:4297 , piece_id=PTHXANV2VCJABYLKRWROSZBDVZZN7TZJIMK4GXWWS5WSF6J7ZO6A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.841839132 (106h23m50.272919253s) [noise=true , address=104.36.82.62:4440 , piece_id=NNRH5KZNUVQQMMB7Q532YKHYVILFLXUSGXPHKS6Q2H4ISM6BU55Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.860024787 (106h23m50.291104918s) [noise=true , address=104.36.82.36:4648 , piece_id=M5PJF2D2QSL6FFJMCONFLCUY7GQCLONGV5XAMKJPSRVAUXXAHIKQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.879951554 (106h23m50.311031675s) [noise=true , address=104.36.82.60:4353 , piece_id=45W3E35Z3NWEJY4RYMSF6XLOB2IMSYTA2LPGW37JCV3OP5YODWWA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.879986100 (106h23m50.311066231s) [seg_pos={0 0}, upload=110735]: Segment reached optimal threshold of 14 pieces. Aug 13 03:49:55.880105158 (106h23m50.311185289s) [noise=true , address=104.36.82.23:4475 , piece_id=GK7HYW44D6DIMEYX4KCPS6JHRQTIEDW2ROEI3MHA7EI2KXM555KQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880168159 (106h23m50.31124829s) [noise=true , address=79.127.201.215:4349 , piece_id=LF3BZJW4MY36VLQGMKZ3TYC2BHWFOPHO4DVAUE5CUDV4RN2CXX7A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880217973 (106h23m50.311298104s) [noise=true , address=104.36.82.23:4451 , piece_id=IPJH7NABQEDR2A5AQ2VYYMBVMZI5BUHGZTF4KP6M4GTSIKQOPR3A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bC4P3xBiPFkxLvavGFVbWKnbprCStL6UJgfKSA2Wd2fX7Mbfw): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880250767 (106h23m50.311330898s) [noise=true , address=79.127.219.38:4036 , piece_id=5JVQOKDLXPABT3CCZBCM7DGXVZYKWMWAQHPETXPA2PSQ4UN5SLAA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DgDXbxPwDCoCgvrRwPxtopfsU5GzqqCtmKRHRVZmVndb1wVrm): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880282638 (106h23m50.311362769s) [seg_pos={0 0}, upload=110735]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:49:55.880377539 (106h23m50.31145766s) [upload=110735]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:49:55.880382067 (106h23m50.311462198s) [upload=110735]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:49:56.200392606 (106h23m50.631472737s) [upload=110735]: Done uploading segments... Aug 13 03:49:56.200410511 (106h23m50.631490642s) [upload=110735]: Upload finished. err: <nil> Duration: 5.675505573s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X.....XX.......................................] duration: 9.002833787s FAIL (2025-08-13 03:50:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.002849416s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X.....XX......................................] duration: 7.752134987s FAIL (2025-08-13 03:50:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.752175586s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X...................X............................X] duration: 4.008436942s FAIL (2025-08-13 03:49:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:49:53.308968726 (106h23m47.740048857s) [upload=110735]: Starting upload Aug 13 03:49:53.309015837 (106h23m47.740095968s) [upload=110735]: Done starting upload Aug 13 03:49:53.309101450 (106h23m47.740181612s) [upload=110735]: Uploading segments... Aug 13 03:49:53.309234645 (106h23m47.740314807s) [upload=110735]: Got next segment. Inline: false Aug 13 03:49:53.309324146 (106h23m47.740404328s) [upload=110735]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:49:53.309331712 (106h23m47.740411893s) [upload=110735]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:49:53.766008247 (106h23m48.197088378s) [seg_pos={0 0}, upload=110735]: Begin upload segment... Aug 13 03:49:53.766052361 (106h23m48.197132482s) [seg_pos={0 0}, upload=110735]: Done begin upload segment. Aug 13 03:49:53.766070065 (106h23m48.197150196s) [noise=true , address=104.36.82.5:4853 , piece_id=TQF7HVLW7FY2OL5Z4CPV3OFFBMPDQUFP5A25CJW6UR7NLHBKAIBA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766080134 (106h23m48.197160255s) [noise=true , address=104.36.82.42:4804 , piece_id=EMN2BEBRBPKDXECQUD7KCGQGYN2EEXINWILWEFISLQC23OHNRB4Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766457186 (106h23m48.197537317s) [noise=true , address=104.36.82.36:4648 , piece_id=M5PJF2D2QSL6FFJMCONFLCUY7GQCLONGV5XAMKJPSRVAUXXAHIKQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766492693 (106h23m48.197572814s) [noise=true , address=104.36.82.35:4602 , piece_id=ZXQQNNRNWYUIRDK7PAGSRGS2QPB2AEFBD6GUEOFY7DFJDO6HVJDA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.766771638 (106h23m48.197851769s) [noise=true , address=104.36.82.11:4148 , piece_id=NJM4JWKBUBEALTKWNO5OXT4OAC2UABQ5KHUIFTQFVHPT2H6IPX3Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.767178185 (106h23m48.198258316s) [noise=true , address=104.36.82.23:4451 , piece_id=IPJH7NABQEDR2A5AQ2VYYMBVMZI5BUHGZTF4KP6M4GTSIKQOPR3A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768376348 (106h23m48.199456479s) [noise=true , address=79.127.219.38:4036 , piece_id=5JVQOKDLXPABT3CCZBCM7DGXVZYKWMWAQHPETXPA2PSQ4UN5SLAA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768871976 (106h23m48.199952107s) [noise=true , address=64.49.13.169:4714 , piece_id=VXCS4DXR3VJCGXL4F4KZXBHN7PPJUVZITJ3HYRN6SPWIPYXSMYPA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.768886523 (106h23m48.199966655s) [noise=true , address=104.36.82.59:4316 , piece_id=UTVPQPZQXEGGN3WTLXSLLG5YKINNKAW3GDYVVGIMETIUWJ722VYA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.769341765 (106h23m48.200421896s) [noise=true , address=77.90.31.18:4822 , piece_id=4ICKC4PJKD6H4CNOAS4WOMZKEFDJ6JZMRBETBZZMAM5Y3LAGQ5NA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.769732592 (106h23m48.200812724s) [noise=true , address=104.36.82.60:4353 , piece_id=45W3E35Z3NWEJY4RYMSF6XLOB2IMSYTA2LPGW37JCV3OP5YODWWA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770050361 (106h23m48.201130492s) [noise=true , address=104.36.82.23:4475 , piece_id=GK7HYW44D6DIMEYX4KCPS6JHRQTIEDW2ROEI3MHA7EI2KXM555KQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770315267 (106h23m48.201395398s) [noise=true , address=104.36.82.58:4297 , piece_id=PTHXANV2VCJABYLKRWROSZBDVZZN7TZJIMK4GXWWS5WSF6J7ZO6A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770557250 (106h23m48.201637371s) [noise=true , address=79.127.201.215:4349 , piece_id=LF3BZJW4MY36VLQGMKZ3TYC2BHWFOPHO4DVAUE5CUDV4RN2CXX7A, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.770700696 (106h23m48.201780837s) [noise=true , address=104.36.82.37:4682 , piece_id=ZXQFTSV24YGFLNKIMYCDVGMDPFGQZAIW6QPC2Z5WWRU4LMWP2UVA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771050183 (106h23m48.202130315s) [noise=true , address=104.36.82.62:4440 , piece_id=NNRH5KZNUVQQMMB7Q532YKHYVILFLXUSGXPHKS6Q2H4ISM6BU55Q, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771365317 (106h23m48.202445448s) [noise=true , address=77.90.31.12:4618 , piece_id=K2VN2PA3EG6YQL4Q4YBMBQEGC4SOSYDAF57LO3YFMYQ37RROUXOQ, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:53.771616087 (106h23m48.202696218s) [noise=true , address=77.90.31.13:4643 , piece_id=YP4TPPDWZ7II2YCKOZOEQGVQMNVUZ74E27YBOKSSC6RICQWCCECA, seg_pos={0 0}, upload=110735]: Uploading piece... Aug 13 03:49:54.879746695 (106h23m49.310826816s) [upload=110735]: Next returned nil segment Aug 13 03:49:54.879761273 (106h23m49.310841394s) [upload=110735]: Waiting for error group managing segments... Aug 13 03:49:55.174383545 (106h23m49.605463676s) [noise=true , address=64.49.13.169:4714 , piece_id=VXCS4DXR3VJCGXL4F4KZXBHN7PPJUVZITJ3HYRN6SPWIPYXSMYPA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.726978445 (106h23m50.158058576s) [noise=true , address=104.36.82.35:4602 , piece_id=ZXQQNNRNWYUIRDK7PAGSRGS2QPB2AEFBD6GUEOFY7DFJDO6HVJDA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.730580046 (106h23m50.161660167s) [noise=true , address=77.90.31.13:4643 , piece_id=YP4TPPDWZ7II2YCKOZOEQGVQMNVUZ74E27YBOKSSC6RICQWCCECA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.733558084 (106h23m50.164638215s) [noise=true , address=104.36.82.37:4682 , piece_id=ZXQFTSV24YGFLNKIMYCDVGMDPFGQZAIW6QPC2Z5WWRU4LMWP2UVA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.755909440 (106h23m50.186989571s) [noise=true , address=104.36.82.5:4853 , piece_id=TQF7HVLW7FY2OL5Z4CPV3OFFBMPDQUFP5A25CJW6UR7NLHBKAIBA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.758609305 (106h23m50.189689436s) [noise=true , address=77.90.31.12:4618 , piece_id=K2VN2PA3EG6YQL4Q4YBMBQEGC4SOSYDAF57LO3YFMYQ37RROUXOQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.760733891 (106h23m50.191814022s) [noise=true , address=104.36.82.59:4316 , piece_id=UTVPQPZQXEGGN3WTLXSLLG5YKINNKAW3GDYVVGIMETIUWJ722VYA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.766054742 (106h23m50.197134873s) [noise=true , address=77.90.31.18:4822 , piece_id=4ICKC4PJKD6H4CNOAS4WOMZKEFDJ6JZMRBETBZZMAM5Y3LAGQ5NA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.781322423 (106h23m50.212402604s) [noise=true , address=104.36.82.42:4804 , piece_id=EMN2BEBRBPKDXECQUD7KCGQGYN2EEXINWILWEFISLQC23OHNRB4Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.803735116 (106h23m50.234815247s) [noise=true , address=104.36.82.11:4148 , piece_id=NJM4JWKBUBEALTKWNO5OXT4OAC2UABQ5KHUIFTQFVHPT2H6IPX3Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.814330439 (106h23m50.24541057s) [noise=true , address=104.36.82.58:4297 , piece_id=PTHXANV2VCJABYLKRWROSZBDVZZN7TZJIMK4GXWWS5WSF6J7ZO6A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.841839132 (106h23m50.272919253s) [noise=true , address=104.36.82.62:4440 , piece_id=NNRH5KZNUVQQMMB7Q532YKHYVILFLXUSGXPHKS6Q2H4ISM6BU55Q, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.860024787 (106h23m50.291104918s) [noise=true , address=104.36.82.36:4648 , piece_id=M5PJF2D2QSL6FFJMCONFLCUY7GQCLONGV5XAMKJPSRVAUXXAHIKQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.879951554 (106h23m50.311031675s) [noise=true , address=104.36.82.60:4353 , piece_id=45W3E35Z3NWEJY4RYMSF6XLOB2IMSYTA2LPGW37JCV3OP5YODWWA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: <nil> Aug 13 03:49:55.879986100 (106h23m50.311066231s) [seg_pos={0 0}, upload=110735]: Segment reached optimal threshold of 14 pieces. Aug 13 03:49:55.880105158 (106h23m50.311185289s) [noise=true , address=104.36.82.23:4475 , piece_id=GK7HYW44D6DIMEYX4KCPS6JHRQTIEDW2ROEI3MHA7EI2KXM555KQ, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UYjE6PEB9gZKwTKcnx23ykF4jDMJzLQoKmiFiejWew5mirdCj): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880168159 (106h23m50.31124829s) [noise=true , address=79.127.201.215:4349 , piece_id=LF3BZJW4MY36VLQGMKZ3TYC2BHWFOPHO4DVAUE5CUDV4RN2CXX7A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880217973 (106h23m50.311298104s) [noise=true , address=104.36.82.23:4451 , piece_id=IPJH7NABQEDR2A5AQ2VYYMBVMZI5BUHGZTF4KP6M4GTSIKQOPR3A, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bC4P3xBiPFkxLvavGFVbWKnbprCStL6UJgfKSA2Wd2fX7Mbfw): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880250767 (106h23m50.311330898s) [noise=true , address=79.127.219.38:4036 , piece_id=5JVQOKDLXPABT3CCZBCM7DGXVZYKWMWAQHPETXPA2PSQ4UN5SLAA, seg_pos={0 0}, upload=110735]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DgDXbxPwDCoCgvrRwPxtopfsU5GzqqCtmKRHRVZmVndb1wVrm): protocol: expected piece hash; context canceled; EOF Aug 13 03:49:55.880282638 (106h23m50.311362769s) [seg_pos={0 0}, upload=110735]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:49:55.880377539 (106h23m50.31145766s) [upload=110735]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:49:55.880382067 (106h23m50.311462198s) [upload=110735]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:49:56.200392606 (106h23m50.631472737s) [upload=110735]: Done uploading segments... Aug 13 03:49:56.200410511 (106h23m50.631490642s) [upload=110735]: Upload finished. err: <nil> Duration: 4.008459695s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.976387467s FAIL (2025-08-13 03:49:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.976419728s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X.....XX.....................................] duration: 7.668007304s FAIL (2025-08-13 03:49:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.66803114s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.896313828s FAIL (2025-08-13 03:48:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.896345186s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X...................X............................] duration: 5.87702783s FAIL (2025-08-13 03:48:48 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:28:24.870384288 (106h2m19.301464419s) [upload=110366]: Starting upload Aug 13 03:28:24.870426058 (106h2m19.30150618s) [upload=110366]: Done starting upload Aug 13 03:28:24.870488848 (106h2m19.301568979s) [upload=110366]: Uploading segments... Aug 13 03:28:24.870604439 (106h2m19.30168457s) [upload=110366]: Got next segment. Inline: false Aug 13 03:28:24.870637743 (106h2m19.301717874s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:28:24.870642031 (106h2m19.301722162s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:28:25.321382698 (106h2m19.752462879s) [seg_pos={0 0}, upload=110366]: Begin upload segment... Aug 13 03:28:25.321441521 (106h2m19.752521652s) [seg_pos={0 0}, upload=110366]: Done begin upload segment. Aug 13 03:28:25.321465406 (106h2m19.752545527s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321492969 (106h2m19.75257309s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321866553 (106h2m19.752946694s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322159584 (106h2m19.753239715s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322826730 (106h2m19.753906862s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322949396 (106h2m19.754029517s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323182792 (106h2m19.754262923s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323412270 (106h2m19.754492402s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323740879 (106h2m19.75482101s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324341969 (106h2m19.755422101s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324756232 (106h2m19.755836363s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325088147 (106h2m19.756168278s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325336562 (106h2m19.756416693s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325678096 (106h2m19.756758217s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325997056 (106h2m19.757077187s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326218289 (106h2m19.75729842s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326380920 (106h2m19.757461051s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326717245 (106h2m19.757797376s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:26.509048537 (106h2m20.940128668s) [upload=110366]: Next returned nil segment Aug 13 03:28:26.509066711 (106h2m20.940146842s) [upload=110366]: Waiting for error group managing segments... Aug 13 03:28:26.809943799 (106h2m21.24102393s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:26.884546048 (106h2m21.315626179s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.268324420 (106h2m21.699404611s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.288897151 (106h2m21.719977283s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.298496038 (106h2m21.729576169s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.305049777 (106h2m21.736129908s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.319397510 (106h2m21.750477631s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.324171042 (106h2m21.755251173s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.325044093 (106h2m21.756124224s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.326189485 (106h2m21.757269616s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.358731589 (106h2m21.78981172s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.364728292 (106h2m21.795808413s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.386279758 (106h2m21.817359889s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393790007 (106h2m21.824870158s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393863398 (106h2m21.824943529s) [seg_pos={0 0}, upload=110366]: Segment reached optimal threshold of 14 pieces. Aug 13 03:28:27.393993957 (106h2m21.825074088s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394051778 (106h2m21.82513191s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394083198 (106h2m21.82516333s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7): protocol: expected piece hash; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; EOF Aug 13 03:28:27.394101453 (106h2m21.825181574s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MScXvmPpGZ6HRRwaoBMrZR3TvhDQh4nZkpSYggRQGzP1JbyVE): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394138204 (106h2m21.825218335s) [seg_pos={0 0}, upload=110366]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:28:27.394212385 (106h2m21.825292516s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:28:27.394217095 (106h2m21.825297226s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:28:27.720702752 (106h2m22.151782883s) [upload=110366]: Done uploading segments... Aug 13 03:28:27.720720255 (106h2m22.151800386s) [upload=110366]: Upload finished. err: <nil> Duration: 5.877047909s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 1.850164917s FAIL (2025-08-13 03:48:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.850184335s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X.....XX....................................] duration: 7.706010705s FAIL (2025-08-13 03:48:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.706031375s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.855931663s FAIL (2025-08-13 03:48:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.855970989s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.881893368s FAIL (2025-08-13 03:47:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.881919047s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 1.857281775s FAIL (2025-08-13 03:47:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.857304829s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X.....XX...................................] duration: 8.171736021s FAIL (2025-08-13 03:47:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.171756159s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X...................X...........................] duration: 5.834271733s FAIL (2025-08-13 03:47:36 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:28:24.870384288 (106h2m19.301464419s) [upload=110366]: Starting upload Aug 13 03:28:24.870426058 (106h2m19.30150618s) [upload=110366]: Done starting upload Aug 13 03:28:24.870488848 (106h2m19.301568979s) [upload=110366]: Uploading segments... Aug 13 03:28:24.870604439 (106h2m19.30168457s) [upload=110366]: Got next segment. Inline: false Aug 13 03:28:24.870637743 (106h2m19.301717874s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:28:24.870642031 (106h2m19.301722162s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:28:25.321382698 (106h2m19.752462879s) [seg_pos={0 0}, upload=110366]: Begin upload segment... Aug 13 03:28:25.321441521 (106h2m19.752521652s) [seg_pos={0 0}, upload=110366]: Done begin upload segment. Aug 13 03:28:25.321465406 (106h2m19.752545527s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321492969 (106h2m19.75257309s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321866553 (106h2m19.752946694s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322159584 (106h2m19.753239715s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322826730 (106h2m19.753906862s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322949396 (106h2m19.754029517s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323182792 (106h2m19.754262923s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323412270 (106h2m19.754492402s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323740879 (106h2m19.75482101s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324341969 (106h2m19.755422101s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324756232 (106h2m19.755836363s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325088147 (106h2m19.756168278s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325336562 (106h2m19.756416693s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325678096 (106h2m19.756758217s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325997056 (106h2m19.757077187s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326218289 (106h2m19.75729842s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326380920 (106h2m19.757461051s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326717245 (106h2m19.757797376s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:26.509048537 (106h2m20.940128668s) [upload=110366]: Next returned nil segment Aug 13 03:28:26.509066711 (106h2m20.940146842s) [upload=110366]: Waiting for error group managing segments... Aug 13 03:28:26.809943799 (106h2m21.24102393s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:26.884546048 (106h2m21.315626179s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.268324420 (106h2m21.699404611s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.288897151 (106h2m21.719977283s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.298496038 (106h2m21.729576169s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.305049777 (106h2m21.736129908s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.319397510 (106h2m21.750477631s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.324171042 (106h2m21.755251173s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.325044093 (106h2m21.756124224s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.326189485 (106h2m21.757269616s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.358731589 (106h2m21.78981172s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.364728292 (106h2m21.795808413s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.386279758 (106h2m21.817359889s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393790007 (106h2m21.824870158s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393863398 (106h2m21.824943529s) [seg_pos={0 0}, upload=110366]: Segment reached optimal threshold of 14 pieces. Aug 13 03:28:27.393993957 (106h2m21.825074088s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394051778 (106h2m21.82513191s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394083198 (106h2m21.82516333s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7): protocol: expected piece hash; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; EOF Aug 13 03:28:27.394101453 (106h2m21.825181574s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MScXvmPpGZ6HRRwaoBMrZR3TvhDQh4nZkpSYggRQGzP1JbyVE): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394138204 (106h2m21.825218335s) [seg_pos={0 0}, upload=110366]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:28:27.394212385 (106h2m21.825292516s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:28:27.394217095 (106h2m21.825297226s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:28:27.720702752 (106h2m22.151782883s) [upload=110366]: Done uploading segments... Aug 13 03:28:27.720720255 (106h2m22.151800386s) [upload=110366]: Upload finished. err: <nil> Duration: 5.834299959s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X.....XX..................................] duration: 7.046473136s FAIL (2025-08-13 03:47:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.046497371s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 1.874680516s FAIL (2025-08-13 03:47:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.874712117s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X.....XX.................................] duration: 7.344261276s FAIL (2025-08-13 03:46:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.344280503s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X...................X..........................] duration: 6.268896999s FAIL (2025-08-13 03:46:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:28:24.870384288 (106h2m19.301464419s) [upload=110366]: Starting upload Aug 13 03:28:24.870426058 (106h2m19.30150618s) [upload=110366]: Done starting upload Aug 13 03:28:24.870488848 (106h2m19.301568979s) [upload=110366]: Uploading segments... Aug 13 03:28:24.870604439 (106h2m19.30168457s) [upload=110366]: Got next segment. Inline: false Aug 13 03:28:24.870637743 (106h2m19.301717874s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:28:24.870642031 (106h2m19.301722162s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:28:25.321382698 (106h2m19.752462879s) [seg_pos={0 0}, upload=110366]: Begin upload segment... Aug 13 03:28:25.321441521 (106h2m19.752521652s) [seg_pos={0 0}, upload=110366]: Done begin upload segment. Aug 13 03:28:25.321465406 (106h2m19.752545527s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321492969 (106h2m19.75257309s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321866553 (106h2m19.752946694s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322159584 (106h2m19.753239715s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322826730 (106h2m19.753906862s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322949396 (106h2m19.754029517s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323182792 (106h2m19.754262923s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323412270 (106h2m19.754492402s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323740879 (106h2m19.75482101s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324341969 (106h2m19.755422101s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324756232 (106h2m19.755836363s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325088147 (106h2m19.756168278s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325336562 (106h2m19.756416693s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325678096 (106h2m19.756758217s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325997056 (106h2m19.757077187s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326218289 (106h2m19.75729842s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326380920 (106h2m19.757461051s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326717245 (106h2m19.757797376s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:26.509048537 (106h2m20.940128668s) [upload=110366]: Next returned nil segment Aug 13 03:28:26.509066711 (106h2m20.940146842s) [upload=110366]: Waiting for error group managing segments... Aug 13 03:28:26.809943799 (106h2m21.24102393s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:26.884546048 (106h2m21.315626179s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.268324420 (106h2m21.699404611s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.288897151 (106h2m21.719977283s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.298496038 (106h2m21.729576169s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.305049777 (106h2m21.736129908s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.319397510 (106h2m21.750477631s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.324171042 (106h2m21.755251173s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.325044093 (106h2m21.756124224s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.326189485 (106h2m21.757269616s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.358731589 (106h2m21.78981172s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.364728292 (106h2m21.795808413s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.386279758 (106h2m21.817359889s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393790007 (106h2m21.824870158s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393863398 (106h2m21.824943529s) [seg_pos={0 0}, upload=110366]: Segment reached optimal threshold of 14 pieces. Aug 13 03:28:27.393993957 (106h2m21.825074088s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394051778 (106h2m21.82513191s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394083198 (106h2m21.82516333s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7): protocol: expected piece hash; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; EOF Aug 13 03:28:27.394101453 (106h2m21.825181574s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MScXvmPpGZ6HRRwaoBMrZR3TvhDQh4nZkpSYggRQGzP1JbyVE): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394138204 (106h2m21.825218335s) [seg_pos={0 0}, upload=110366]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:28:27.394212385 (106h2m21.825292516s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:28:27.394217095 (106h2m21.825297226s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:28:27.720702752 (106h2m22.151782883s) [upload=110366]: Done uploading segments... Aug 13 03:28:27.720720255 (106h2m22.151800386s) [upload=110366]: Upload finished. err: <nil> Duration: 6.268947576s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.879326417s FAIL (2025-08-13 03:46:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.879346765s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 2.074044419s FAIL (2025-08-13 03:46:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.074068083s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X...................X.........................] duration: 5.637890759s FAIL (2025-08-13 03:46:04 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:28:24.870384288 (106h2m19.301464419s) [upload=110366]: Starting upload Aug 13 03:28:24.870426058 (106h2m19.30150618s) [upload=110366]: Done starting upload Aug 13 03:28:24.870488848 (106h2m19.301568979s) [upload=110366]: Uploading segments... Aug 13 03:28:24.870604439 (106h2m19.30168457s) [upload=110366]: Got next segment. Inline: false Aug 13 03:28:24.870637743 (106h2m19.301717874s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:28:24.870642031 (106h2m19.301722162s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:28:25.321382698 (106h2m19.752462879s) [seg_pos={0 0}, upload=110366]: Begin upload segment... Aug 13 03:28:25.321441521 (106h2m19.752521652s) [seg_pos={0 0}, upload=110366]: Done begin upload segment. Aug 13 03:28:25.321465406 (106h2m19.752545527s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321492969 (106h2m19.75257309s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321866553 (106h2m19.752946694s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322159584 (106h2m19.753239715s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322826730 (106h2m19.753906862s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322949396 (106h2m19.754029517s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323182792 (106h2m19.754262923s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323412270 (106h2m19.754492402s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323740879 (106h2m19.75482101s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324341969 (106h2m19.755422101s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324756232 (106h2m19.755836363s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325088147 (106h2m19.756168278s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325336562 (106h2m19.756416693s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325678096 (106h2m19.756758217s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325997056 (106h2m19.757077187s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326218289 (106h2m19.75729842s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326380920 (106h2m19.757461051s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326717245 (106h2m19.757797376s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:26.509048537 (106h2m20.940128668s) [upload=110366]: Next returned nil segment Aug 13 03:28:26.509066711 (106h2m20.940146842s) [upload=110366]: Waiting for error group managing segments... Aug 13 03:28:26.809943799 (106h2m21.24102393s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:26.884546048 (106h2m21.315626179s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.268324420 (106h2m21.699404611s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.288897151 (106h2m21.719977283s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.298496038 (106h2m21.729576169s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.305049777 (106h2m21.736129908s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.319397510 (106h2m21.750477631s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.324171042 (106h2m21.755251173s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.325044093 (106h2m21.756124224s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.326189485 (106h2m21.757269616s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.358731589 (106h2m21.78981172s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.364728292 (106h2m21.795808413s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.386279758 (106h2m21.817359889s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393790007 (106h2m21.824870158s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393863398 (106h2m21.824943529s) [seg_pos={0 0}, upload=110366]: Segment reached optimal threshold of 14 pieces. Aug 13 03:28:27.393993957 (106h2m21.825074088s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394051778 (106h2m21.82513191s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394083198 (106h2m21.82516333s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7): protocol: expected piece hash; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; EOF Aug 13 03:28:27.394101453 (106h2m21.825181574s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MScXvmPpGZ6HRRwaoBMrZR3TvhDQh4nZkpSYggRQGzP1JbyVE): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394138204 (106h2m21.825218335s) [seg_pos={0 0}, upload=110366]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:28:27.394212385 (106h2m21.825292516s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:28:27.394217095 (106h2m21.825297226s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:28:27.720702752 (106h2m22.151782883s) [upload=110366]: Done uploading segments... Aug 13 03:28:27.720720255 (106h2m22.151800386s) [upload=110366]: Upload finished. err: <nil> Duration: 5.637919373s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X.....XX................................] duration: 9.344555169s FAIL (2025-08-13 03:46:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.344579296s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.925728991s FAIL (2025-08-13 03:45:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.925753219s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.896654712s FAIL (2025-08-13 03:45:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.896675661s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X...................X........................] duration: 5.878943044s FAIL (2025-08-13 03:45:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:28:24.870384288 (106h2m19.301464419s) [upload=110366]: Starting upload Aug 13 03:28:24.870426058 (106h2m19.30150618s) [upload=110366]: Done starting upload Aug 13 03:28:24.870488848 (106h2m19.301568979s) [upload=110366]: Uploading segments... Aug 13 03:28:24.870604439 (106h2m19.30168457s) [upload=110366]: Got next segment. Inline: false Aug 13 03:28:24.870637743 (106h2m19.301717874s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:28:24.870642031 (106h2m19.301722162s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:28:25.321382698 (106h2m19.752462879s) [seg_pos={0 0}, upload=110366]: Begin upload segment... Aug 13 03:28:25.321441521 (106h2m19.752521652s) [seg_pos={0 0}, upload=110366]: Done begin upload segment. Aug 13 03:28:25.321465406 (106h2m19.752545527s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321492969 (106h2m19.75257309s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321866553 (106h2m19.752946694s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322159584 (106h2m19.753239715s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322826730 (106h2m19.753906862s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322949396 (106h2m19.754029517s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323182792 (106h2m19.754262923s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323412270 (106h2m19.754492402s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323740879 (106h2m19.75482101s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324341969 (106h2m19.755422101s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324756232 (106h2m19.755836363s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325088147 (106h2m19.756168278s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325336562 (106h2m19.756416693s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325678096 (106h2m19.756758217s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325997056 (106h2m19.757077187s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326218289 (106h2m19.75729842s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326380920 (106h2m19.757461051s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326717245 (106h2m19.757797376s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:26.509048537 (106h2m20.940128668s) [upload=110366]: Next returned nil segment Aug 13 03:28:26.509066711 (106h2m20.940146842s) [upload=110366]: Waiting for error group managing segments... Aug 13 03:28:26.809943799 (106h2m21.24102393s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:26.884546048 (106h2m21.315626179s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.268324420 (106h2m21.699404611s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.288897151 (106h2m21.719977283s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.298496038 (106h2m21.729576169s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.305049777 (106h2m21.736129908s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.319397510 (106h2m21.750477631s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.324171042 (106h2m21.755251173s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.325044093 (106h2m21.756124224s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.326189485 (106h2m21.757269616s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.358731589 (106h2m21.78981172s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.364728292 (106h2m21.795808413s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.386279758 (106h2m21.817359889s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393790007 (106h2m21.824870158s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393863398 (106h2m21.824943529s) [seg_pos={0 0}, upload=110366]: Segment reached optimal threshold of 14 pieces. Aug 13 03:28:27.393993957 (106h2m21.825074088s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394051778 (106h2m21.82513191s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394083198 (106h2m21.82516333s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7): protocol: expected piece hash; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; EOF Aug 13 03:28:27.394101453 (106h2m21.825181574s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MScXvmPpGZ6HRRwaoBMrZR3TvhDQh4nZkpSYggRQGzP1JbyVE): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394138204 (106h2m21.825218335s) [seg_pos={0 0}, upload=110366]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:28:27.394212385 (106h2m21.825292516s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:28:27.394217095 (106h2m21.825297226s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:28:27.720702752 (106h2m22.151782883s) [upload=110366]: Done uploading segments... Aug 13 03:28:27.720720255 (106h2m22.151800386s) [upload=110366]: Upload finished. err: <nil> Duration: 5.87896677s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.961146734s FAIL (2025-08-13 03:45:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.961176863s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X.....XX...............................] duration: 7.29334687s FAIL (2025-08-13 03:45:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.293373662s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [X...............X...................X.......................] duration: 5.960541997s FAIL (2025-08-13 03:44:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:28:24.870384288 (106h2m19.301464419s) [upload=110366]: Starting upload Aug 13 03:28:24.870426058 (106h2m19.30150618s) [upload=110366]: Done starting upload Aug 13 03:28:24.870488848 (106h2m19.301568979s) [upload=110366]: Uploading segments... Aug 13 03:28:24.870604439 (106h2m19.30168457s) [upload=110366]: Got next segment. Inline: false Aug 13 03:28:24.870637743 (106h2m19.301717874s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:28:24.870642031 (106h2m19.301722162s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:28:25.321382698 (106h2m19.752462879s) [seg_pos={0 0}, upload=110366]: Begin upload segment... Aug 13 03:28:25.321441521 (106h2m19.752521652s) [seg_pos={0 0}, upload=110366]: Done begin upload segment. Aug 13 03:28:25.321465406 (106h2m19.752545527s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321492969 (106h2m19.75257309s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321866553 (106h2m19.752946694s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322159584 (106h2m19.753239715s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322826730 (106h2m19.753906862s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322949396 (106h2m19.754029517s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323182792 (106h2m19.754262923s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323412270 (106h2m19.754492402s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323740879 (106h2m19.75482101s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324341969 (106h2m19.755422101s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324756232 (106h2m19.755836363s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325088147 (106h2m19.756168278s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325336562 (106h2m19.756416693s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325678096 (106h2m19.756758217s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325997056 (106h2m19.757077187s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326218289 (106h2m19.75729842s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326380920 (106h2m19.757461051s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326717245 (106h2m19.757797376s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:26.509048537 (106h2m20.940128668s) [upload=110366]: Next returned nil segment Aug 13 03:28:26.509066711 (106h2m20.940146842s) [upload=110366]: Waiting for error group managing segments... Aug 13 03:28:26.809943799 (106h2m21.24102393s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:26.884546048 (106h2m21.315626179s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.268324420 (106h2m21.699404611s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.288897151 (106h2m21.719977283s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.298496038 (106h2m21.729576169s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.305049777 (106h2m21.736129908s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.319397510 (106h2m21.750477631s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.324171042 (106h2m21.755251173s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.325044093 (106h2m21.756124224s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.326189485 (106h2m21.757269616s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.358731589 (106h2m21.78981172s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.364728292 (106h2m21.795808413s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.386279758 (106h2m21.817359889s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393790007 (106h2m21.824870158s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393863398 (106h2m21.824943529s) [seg_pos={0 0}, upload=110366]: Segment reached optimal threshold of 14 pieces. Aug 13 03:28:27.393993957 (106h2m21.825074088s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394051778 (106h2m21.82513191s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394083198 (106h2m21.82516333s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7): protocol: expected piece hash; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; EOF Aug 13 03:28:27.394101453 (106h2m21.825181574s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MScXvmPpGZ6HRRwaoBMrZR3TvhDQh4nZkpSYggRQGzP1JbyVE): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394138204 (106h2m21.825218335s) [seg_pos={0 0}, upload=110366]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:28:27.394212385 (106h2m21.825292516s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:28:27.394217095 (106h2m21.825297226s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:28:27.720702752 (106h2m22.151782883s) [upload=110366]: Done uploading segments... Aug 13 03:28:27.720720255 (106h2m22.151800386s) [upload=110366]: Upload finished. err: <nil> Duration: 5.960567997s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.934419605s FAIL (2025-08-13 03:44:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.934447237s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X...............X...................X......................] duration: 5.953877556s FAIL (2025-08-13 03:44:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:28:24.870384288 (106h2m19.301464419s) [upload=110366]: Starting upload Aug 13 03:28:24.870426058 (106h2m19.30150618s) [upload=110366]: Done starting upload Aug 13 03:28:24.870488848 (106h2m19.301568979s) [upload=110366]: Uploading segments... Aug 13 03:28:24.870604439 (106h2m19.30168457s) [upload=110366]: Got next segment. Inline: false Aug 13 03:28:24.870637743 (106h2m19.301717874s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:28:24.870642031 (106h2m19.301722162s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:28:25.321382698 (106h2m19.752462879s) [seg_pos={0 0}, upload=110366]: Begin upload segment... Aug 13 03:28:25.321441521 (106h2m19.752521652s) [seg_pos={0 0}, upload=110366]: Done begin upload segment. Aug 13 03:28:25.321465406 (106h2m19.752545527s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321492969 (106h2m19.75257309s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321866553 (106h2m19.752946694s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322159584 (106h2m19.753239715s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322826730 (106h2m19.753906862s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322949396 (106h2m19.754029517s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323182792 (106h2m19.754262923s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323412270 (106h2m19.754492402s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323740879 (106h2m19.75482101s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324341969 (106h2m19.755422101s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324756232 (106h2m19.755836363s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325088147 (106h2m19.756168278s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325336562 (106h2m19.756416693s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325678096 (106h2m19.756758217s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325997056 (106h2m19.757077187s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326218289 (106h2m19.75729842s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326380920 (106h2m19.757461051s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326717245 (106h2m19.757797376s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:26.509048537 (106h2m20.940128668s) [upload=110366]: Next returned nil segment Aug 13 03:28:26.509066711 (106h2m20.940146842s) [upload=110366]: Waiting for error group managing segments... Aug 13 03:28:26.809943799 (106h2m21.24102393s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:26.884546048 (106h2m21.315626179s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.268324420 (106h2m21.699404611s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.288897151 (106h2m21.719977283s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.298496038 (106h2m21.729576169s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.305049777 (106h2m21.736129908s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.319397510 (106h2m21.750477631s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.324171042 (106h2m21.755251173s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.325044093 (106h2m21.756124224s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.326189485 (106h2m21.757269616s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.358731589 (106h2m21.78981172s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.364728292 (106h2m21.795808413s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.386279758 (106h2m21.817359889s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393790007 (106h2m21.824870158s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393863398 (106h2m21.824943529s) [seg_pos={0 0}, upload=110366]: Segment reached optimal threshold of 14 pieces. Aug 13 03:28:27.393993957 (106h2m21.825074088s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394051778 (106h2m21.82513191s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394083198 (106h2m21.82516333s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7): protocol: expected piece hash; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; EOF Aug 13 03:28:27.394101453 (106h2m21.825181574s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MScXvmPpGZ6HRRwaoBMrZR3TvhDQh4nZkpSYggRQGzP1JbyVE): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394138204 (106h2m21.825218335s) [seg_pos={0 0}, upload=110366]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:28:27.394212385 (106h2m21.825292516s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:28:27.394217095 (106h2m21.825297226s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:28:27.720702752 (106h2m22.151782883s) [upload=110366]: Done uploading segments... Aug 13 03:28:27.720720255 (106h2m22.151800386s) [upload=110366]: Upload finished. err: <nil> Duration: 5.953904747s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.927438371s FAIL (2025-08-13 03:44:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.927466515s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.02114791s FAIL (2025-08-13 03:44:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 2.021176393s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X.....XX..............................] duration: 7.646829065s FAIL (2025-08-13 03:43:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.646842641s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.893013232s FAIL (2025-08-13 03:43:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 13 03:06:41.898033585 (105h40m36.329113716s) [upload=110007]: Starting upload Aug 13 03:06:41.898093971 (105h40m36.329174102s) [upload=110007]: Done starting upload Aug 13 03:06:41.898140650 (105h40m36.329220781s) [upload=110007]: Uploading segments... Aug 13 03:06:41.898172331 (105h40m36.329252462s) [upload=110007]: Got next segment. Inline: true Aug 13 03:06:41.898176078 (105h40m36.329256209s) [upload=110007]: Waiting for error group managing segments... Aug 13 03:06:41.898208831 (105h40m36.329288962s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:06:41.898212778 (105h40m36.329292909s) [upload=110007]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:06:41.898215563 (105h40m36.329295694s) [upload=110007]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:07:11.244546166 (105h41m5.675626307s) [upload=110007]: Done uploading segments... Aug 13 03:07:11.244585330 (105h41m5.675665482s) [upload=110007]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.893036177s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X...............X...................X.....................] duration: 5.837753784s FAIL (2025-08-13 03:43:45 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:28:24.870384288 (106h2m19.301464419s) [upload=110366]: Starting upload Aug 13 03:28:24.870426058 (106h2m19.30150618s) [upload=110366]: Done starting upload Aug 13 03:28:24.870488848 (106h2m19.301568979s) [upload=110366]: Uploading segments... Aug 13 03:28:24.870604439 (106h2m19.30168457s) [upload=110366]: Got next segment. Inline: false Aug 13 03:28:24.870637743 (106h2m19.301717874s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:28:24.870642031 (106h2m19.301722162s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:28:25.321382698 (106h2m19.752462879s) [seg_pos={0 0}, upload=110366]: Begin upload segment... Aug 13 03:28:25.321441521 (106h2m19.752521652s) [seg_pos={0 0}, upload=110366]: Done begin upload segment. Aug 13 03:28:25.321465406 (106h2m19.752545527s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321492969 (106h2m19.75257309s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321866553 (106h2m19.752946694s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322159584 (106h2m19.753239715s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322826730 (106h2m19.753906862s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322949396 (106h2m19.754029517s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323182792 (106h2m19.754262923s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323412270 (106h2m19.754492402s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323740879 (106h2m19.75482101s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324341969 (106h2m19.755422101s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324756232 (106h2m19.755836363s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325088147 (106h2m19.756168278s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325336562 (106h2m19.756416693s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325678096 (106h2m19.756758217s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325997056 (106h2m19.757077187s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326218289 (106h2m19.75729842s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326380920 (106h2m19.757461051s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326717245 (106h2m19.757797376s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:26.509048537 (106h2m20.940128668s) [upload=110366]: Next returned nil segment Aug 13 03:28:26.509066711 (106h2m20.940146842s) [upload=110366]: Waiting for error group managing segments... Aug 13 03:28:26.809943799 (106h2m21.24102393s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:26.884546048 (106h2m21.315626179s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.268324420 (106h2m21.699404611s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.288897151 (106h2m21.719977283s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.298496038 (106h2m21.729576169s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.305049777 (106h2m21.736129908s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.319397510 (106h2m21.750477631s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.324171042 (106h2m21.755251173s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.325044093 (106h2m21.756124224s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.326189485 (106h2m21.757269616s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.358731589 (106h2m21.78981172s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.364728292 (106h2m21.795808413s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.386279758 (106h2m21.817359889s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393790007 (106h2m21.824870158s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393863398 (106h2m21.824943529s) [seg_pos={0 0}, upload=110366]: Segment reached optimal threshold of 14 pieces. Aug 13 03:28:27.393993957 (106h2m21.825074088s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394051778 (106h2m21.82513191s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394083198 (106h2m21.82516333s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7): protocol: expected piece hash; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; EOF Aug 13 03:28:27.394101453 (106h2m21.825181574s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MScXvmPpGZ6HRRwaoBMrZR3TvhDQh4nZkpSYggRQGzP1JbyVE): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394138204 (106h2m21.825218335s) [seg_pos={0 0}, upload=110366]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:28:27.394212385 (106h2m21.825292516s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:28:27.394217095 (106h2m21.825297226s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:28:27.720702752 (106h2m22.151782883s) [upload=110366]: Done uploading segments... Aug 13 03:28:27.720720255 (106h2m22.151800386s) [upload=110366]: Upload finished. err: <nil> Duration: 5.837779294s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 1.955199067s FAIL (2025-08-13 03:43:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 13 03:23:34.987469695 (105h57m29.418549826s) [upload=110284]: Starting upload Aug 13 03:23:34.987520002 (105h57m29.418600133s) [upload=110284]: Done starting upload Aug 13 03:23:34.987570067 (105h57m29.418650198s) [upload=110284]: Uploading segments... Aug 13 03:23:34.987604243 (105h57m29.418684374s) [upload=110284]: Got next segment. Inline: true Aug 13 03:23:34.987608641 (105h57m29.418688772s) [upload=110284]: Waiting for error group managing segments... Aug 13 03:23:34.987640973 (105h57m29.418721094s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:23:34.987644710 (105h57m29.418724841s) [upload=110284]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 03:23:34.987647044 (105h57m29.418727165s) [upload=110284]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:23:35.199946062 (105h57m29.631026193s) [upload=110284]: Done uploading segments... Aug 13 03:23:35.199964076 (105h57m29.631044207s) [upload=110284]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.95522636s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X...............X...................X....................] duration: 5.697627514s FAIL (2025-08-13 03:43:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 03:28:24.870384288 (106h2m19.301464419s) [upload=110366]: Starting upload Aug 13 03:28:24.870426058 (106h2m19.30150618s) [upload=110366]: Done starting upload Aug 13 03:28:24.870488848 (106h2m19.301568979s) [upload=110366]: Uploading segments... Aug 13 03:28:24.870604439 (106h2m19.30168457s) [upload=110366]: Got next segment. Inline: false Aug 13 03:28:24.870637743 (106h2m19.301717874s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 03:28:24.870642031 (106h2m19.301722162s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 03:28:25.321382698 (106h2m19.752462879s) [seg_pos={0 0}, upload=110366]: Begin upload segment... Aug 13 03:28:25.321441521 (106h2m19.752521652s) [seg_pos={0 0}, upload=110366]: Done begin upload segment. Aug 13 03:28:25.321465406 (106h2m19.752545527s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321492969 (106h2m19.75257309s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.321866553 (106h2m19.752946694s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322159584 (106h2m19.753239715s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322826730 (106h2m19.753906862s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.322949396 (106h2m19.754029517s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323182792 (106h2m19.754262923s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323412270 (106h2m19.754492402s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.323740879 (106h2m19.75482101s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324341969 (106h2m19.755422101s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.324756232 (106h2m19.755836363s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325088147 (106h2m19.756168278s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325336562 (106h2m19.756416693s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325678096 (106h2m19.756758217s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.325997056 (106h2m19.757077187s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326218289 (106h2m19.75729842s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326380920 (106h2m19.757461051s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:25.326717245 (106h2m19.757797376s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Uploading piece... Aug 13 03:28:26.509048537 (106h2m20.940128668s) [upload=110366]: Next returned nil segment Aug 13 03:28:26.509066711 (106h2m20.940146842s) [upload=110366]: Waiting for error group managing segments... Aug 13 03:28:26.809943799 (106h2m21.24102393s) [noise=true , address=64.49.13.139:4478 , piece_id=ZLJZ3QKNWE7LHFZSZEFOSZ6EN27KIAJFEEY7TVL4YW2VZXQM6IOA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:26.884546048 (106h2m21.315626179s) [noise=true , address=64.49.13.139:4473 , piece_id=GYBREOGJD222LYFTSGTYHYLM4T22ZKSQYKN4GNW42ZQSNZQOHQJA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.268324420 (106h2m21.699404611s) [noise=true , address=104.36.82.38:4716 , piece_id=3Q4BIBCFEYXICWG3CEFEUIBAMBNZWJ7FNCMMB4DHJ73RHKVXGBKQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.288897151 (106h2m21.719977283s) [noise=true , address=104.36.82.58:4298 , piece_id=I76VJFWH36OITPI2LEHTUPWFSC566LKQZBGM7VVJ6744TYN2QESA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.298496038 (106h2m21.729576169s) [noise=true , address=104.36.82.36:4641 , piece_id=3A5FI2NFX2Y2QYX6JMEMIPXAYU6ZGB662F6PZGGY6W456MNN4H2Q, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.305049777 (106h2m21.736129908s) [noise=true , address=104.36.82.35:4585 , piece_id=EA7XTERIGFLJMV33KWPJJEFXKDNYHSVJIBYES33AANWX6RFO3Z4A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.319397510 (106h2m21.750477631s) [noise=true , address=104.36.82.21:4488 , piece_id=ZJC77OAGWGPQHJLXGSY2JUS4POTFI4UYW2ORN2OET4U5QCCK2RYA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.324171042 (106h2m21.755251173s) [noise=true , address=104.36.82.8:4988 , piece_id=KZO4HIEGBYBTXIUMP7YETN5WBBNS2ZMGOA3W6H377RF5RV3RIGRQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.325044093 (106h2m21.756124224s) [noise=true , address=104.36.82.39:4758 , piece_id=6GQJIBOJ6XK6RILWWEPLNQH7OPKSYWFMQG3SXYTVONZRLV3YDU7A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.326189485 (106h2m21.757269616s) [noise=true , address=104.36.82.4:4833 , piece_id=WOJB675NZW5MONNZ7MHECJEYFRFUQDDQFCG4EATF2DTTQOY5FPIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.358731589 (106h2m21.78981172s) [noise=true , address=104.36.82.15:4267 , piece_id=YOBGDLVCGO7Z6XLQQ6K7INK36KDGOHZNA6L2ROJF2VVC3YOE5CEQ, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.364728292 (106h2m21.795808413s) [noise=true , address=104.36.82.37:4686 , piece_id=TWWJUP6LY7VX4BW7PSTTZK53JTGSVBP2Y6NQCR3PXT44WCLGY2WA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.386279758 (106h2m21.817359889s) [noise=true , address=77.90.31.21:4906 , piece_id=FLIZ63F7NOFHDUEDYOAGMMGEGS473UQPXY2C3WKX67MGWJJEZBIA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393790007 (106h2m21.824870158s) [noise=true , address=104.36.82.44:4876 , piece_id=576P5WMAQ6BCLTG4UAJX2ZCHGAMKCD47Z6YQBWJKHI5MQGGXT4XA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: <nil> Aug 13 03:28:27.393863398 (106h2m21.824943529s) [seg_pos={0 0}, upload=110366]: Segment reached optimal threshold of 14 pieces. Aug 13 03:28:27.393993957 (106h2m21.825074088s) [noise=true , address=104.36.82.46:4496 , piece_id=K6J6AEVYWM63WM6IFDXSGBUWK6WX72R26RLVLRZ64VSF4H2744JA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394051778 (106h2m21.82513191s) [noise=true , address=104.36.82.5:4866 , piece_id=22IXIQNEVDD6VILGPIQMS46JFYVQDI3BNI633KABQLCDBQQB22ZA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zUcYfJE8xRW2Tow7bGB3PeNrVBUzVWR82JADkVwPdSr4CTPyr): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394083198 (106h2m21.82516333s) [noise=true , address=104.36.82.60:4363 , piece_id=SRLXSTGJIEBAM4H6UE6AF32SJY7N2ZHJPPDOXKTUN44ZI3LT2I5A, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1endke3P5J92vFytHPiVczH5VcuHDSXyvd4qK9kFNhkQh8MhL7): protocol: expected piece hash; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; write tcp 188.166.225.170:42920->104.36.82.60:4363: use of closed network connection; EOF Aug 13 03:28:27.394101453 (106h2m21.825181574s) [noise=true , address=79.127.220.98:4422 , piece_id=R6WFYRTY4IEQ4ORVBDWP5FBP676QRCJDPKSBNJDXL64C2OH2SWDA, seg_pos={0 0}, upload=110366]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MScXvmPpGZ6HRRwaoBMrZR3TvhDQh4nZkpSYggRQGzP1JbyVE): protocol: expected piece hash; context canceled; EOF Aug 13 03:28:27.394138204 (106h2m21.825218335s) [seg_pos={0 0}, upload=110366]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 03:28:27.394212385 (106h2m21.825292516s) [upload=110366]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 03:28:27.394217095 (106h2m21.825297226s) [upload=110366]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 03:28:27.720702752 (106h2m22.151782883s) [upload=110366]: Done uploading segments... Aug 13 03:28:27.720720255 (106h2m22.151800386s) [upload=110366]: Upload finished. err: <nil> Duration: 5.697655817s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X.....XX.............................] duration: 8.792473248s FAIL (2025-08-13 03:42:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.792494169s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)