Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X.......................X...X.........] duration: 13.225383635s FAIL (2025-08-09 00:57:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 00:48:46.017517689 (7h22m40.44859781s) [upload=7718]: Starting upload Aug 9 00:48:46.017561742 (7h22m40.448641863s) [upload=7718]: Done starting upload Aug 9 00:48:46.017629412 (7h22m40.448709543s) [upload=7718]: Uploading segments... Aug 9 00:48:46.017700498 (7h22m40.448780629s) [upload=7718]: Got next segment. Inline: false Aug 9 00:48:46.017739152 (7h22m40.448819273s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:48:46.017743611 (7h22m40.448823742s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 00:48:46.487633950 (7h22m40.918714081s) [seg_pos={0 0}, upload=7718]: Begin upload segment... Aug 9 00:48:46.487687452 (7h22m40.918767583s) [seg_pos={0 0}, upload=7718]: Done begin upload segment. Aug 9 00:48:46.487711057 (7h22m40.918791188s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.487756925 (7h22m40.918837056s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488097557 (7h22m40.919177688s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488233727 (7h22m40.919313858s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488374266 (7h22m40.919454397s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488628011 (7h22m40.919708142s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488856018 (7h22m40.919936139s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489126416 (7h22m40.920206537s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489348511 (7h22m40.920428642s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489581706 (7h22m40.920661837s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489787100 (7h22m40.920867231s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489998043 (7h22m40.921078174s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490181434 (7h22m40.921261565s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490363022 (7h22m40.921443153s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490552123 (7h22m40.921632254s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490745964 (7h22m40.921826085s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490924466 (7h22m40.922004597s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.491108248 (7h22m40.922188379s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:47.595108697 (7h22m42.026188828s) [upload=7718]: Next returned nil segment Aug 9 00:48:47.595125159 (7h22m42.02620529s) [upload=7718]: Waiting for error group managing segments... Aug 9 00:48:47.877294483 (7h22m42.308374614s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.948937908 (7h22m42.380018039s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.968047280 (7h22m42.399127411s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.520661895 (7h22m42.951742026s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.523938645 (7h22m42.955018776s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.544420141 (7h22m42.975500272s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.566672848 (7h22m42.997752979s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.582416790 (7h22m43.013496921s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.585069256 (7h22m43.016149387s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.658756792 (7h22m43.089836923s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.659623009 (7h22m43.09070314s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.873462627 (7h22m43.304542758s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.259262096 (7h22m43.690342227s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319042805 (7h22m43.750122936s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319066630 (7h22m43.750146761s) [seg_pos={0 0}, upload=7718]: Segment reached optimal threshold of 14 pieces. Aug 9 00:48:49.319407803 (7h22m43.750487934s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: rpc: dial tcp 104.36.82.44:4894: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319452639 (7h22m43.75053277s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UpWmMhEmk5veKZvzsQ3MCxCMnfEKtFgCugBTsYfy9Ejm7G4xD): protocol: read tcp 188.166.225.170:35022->104.36.82.15:4292: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319470654 (7h22m43.750550785s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 9 00:48:49.319573690 (7h22m43.750653821s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b): protocol: write tcp 188.166.225.170:41310->104.36.82.19:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319629567 (7h22m43.750709698s) [seg_pos={0 0}, upload=7718]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 00:48:49.319709189 (7h22m43.75078932s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 00:48:49.319714160 (7h22m43.750794291s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:48:49.635176021 (7h22m44.066256152s) [upload=7718]: Done uploading segments... Aug 9 00:48:49.635194487 (7h22m44.066274618s) [upload=7718]: Upload finished. err: <nil> Duration: 13.225414945s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.481052161s FAIL (2025-08-09 00:57:06 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.481075346s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 762.508103ms FAIL (2025-08-09 00:57:01 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 762.532819ms edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [X.X..........................X.X.....X................X.....] duration: 1.839020476s FAIL (2025-08-09 00:56:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:53:50.823389357 (7h27m45.254469488s) [upload=7803]: Starting upload Aug 9 00:53:50.823435035 (7h27m45.254515166s) [upload=7803]: Done starting upload Aug 9 00:53:50.823486734 (7h27m45.254566855s) [upload=7803]: Uploading segments... Aug 9 00:53:50.823519837 (7h27m45.254599968s) [upload=7803]: Got next segment. Inline: true Aug 9 00:53:50.823534765 (7h27m45.254614896s) [upload=7803]: Waiting for error group managing segments... Aug 9 00:53:50.823567247 (7h27m45.254647378s) [upload=7803]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:53:50.823571045 (7h27m45.254651176s) [upload=7803]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:53:50.823573500 (7h27m45.254653621s) [upload=7803]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:53:51.036378597 (7h27m45.467458728s) [upload=7803]: Done uploading segments... Aug 9 00:53:51.036397072 (7h27m45.467477203s) [upload=7803]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.839048861s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.29985639s FAIL (2025-08-09 00:56:32 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.299878252s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 737.277188ms FAIL (2025-08-09 00:56:09 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 737.312916ms edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X.......................X...X........] duration: 6.880135208s FAIL (2025-08-09 00:56:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 00:48:46.017517689 (7h22m40.44859781s) [upload=7718]: Starting upload Aug 9 00:48:46.017561742 (7h22m40.448641863s) [upload=7718]: Done starting upload Aug 9 00:48:46.017629412 (7h22m40.448709543s) [upload=7718]: Uploading segments... Aug 9 00:48:46.017700498 (7h22m40.448780629s) [upload=7718]: Got next segment. Inline: false Aug 9 00:48:46.017739152 (7h22m40.448819273s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:48:46.017743611 (7h22m40.448823742s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 00:48:46.487633950 (7h22m40.918714081s) [seg_pos={0 0}, upload=7718]: Begin upload segment... Aug 9 00:48:46.487687452 (7h22m40.918767583s) [seg_pos={0 0}, upload=7718]: Done begin upload segment. Aug 9 00:48:46.487711057 (7h22m40.918791188s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.487756925 (7h22m40.918837056s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488097557 (7h22m40.919177688s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488233727 (7h22m40.919313858s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488374266 (7h22m40.919454397s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488628011 (7h22m40.919708142s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488856018 (7h22m40.919936139s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489126416 (7h22m40.920206537s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489348511 (7h22m40.920428642s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489581706 (7h22m40.920661837s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489787100 (7h22m40.920867231s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489998043 (7h22m40.921078174s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490181434 (7h22m40.921261565s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490363022 (7h22m40.921443153s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490552123 (7h22m40.921632254s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490745964 (7h22m40.921826085s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490924466 (7h22m40.922004597s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.491108248 (7h22m40.922188379s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:47.595108697 (7h22m42.026188828s) [upload=7718]: Next returned nil segment Aug 9 00:48:47.595125159 (7h22m42.02620529s) [upload=7718]: Waiting for error group managing segments... Aug 9 00:48:47.877294483 (7h22m42.308374614s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.948937908 (7h22m42.380018039s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.968047280 (7h22m42.399127411s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.520661895 (7h22m42.951742026s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.523938645 (7h22m42.955018776s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.544420141 (7h22m42.975500272s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.566672848 (7h22m42.997752979s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.582416790 (7h22m43.013496921s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.585069256 (7h22m43.016149387s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.658756792 (7h22m43.089836923s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.659623009 (7h22m43.09070314s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.873462627 (7h22m43.304542758s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.259262096 (7h22m43.690342227s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319042805 (7h22m43.750122936s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319066630 (7h22m43.750146761s) [seg_pos={0 0}, upload=7718]: Segment reached optimal threshold of 14 pieces. Aug 9 00:48:49.319407803 (7h22m43.750487934s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: rpc: dial tcp 104.36.82.44:4894: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319452639 (7h22m43.75053277s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UpWmMhEmk5veKZvzsQ3MCxCMnfEKtFgCugBTsYfy9Ejm7G4xD): protocol: read tcp 188.166.225.170:35022->104.36.82.15:4292: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319470654 (7h22m43.750550785s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 9 00:48:49.319573690 (7h22m43.750653821s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b): protocol: write tcp 188.166.225.170:41310->104.36.82.19:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319629567 (7h22m43.750709698s) [seg_pos={0 0}, upload=7718]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 00:48:49.319709189 (7h22m43.75078932s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 00:48:49.319714160 (7h22m43.750794291s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:48:49.635176021 (7h22m44.066256152s) [upload=7718]: Done uploading segments... Aug 9 00:48:49.635194487 (7h22m44.066274618s) [upload=7718]: Upload finished. err: <nil> Duration: 6.88016683s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [.X.X..........................X.X.....X................X....] duration: 1.89425164s FAIL (2025-08-09 00:55:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:53:50.823389357 (7h27m45.254469488s) [upload=7803]: Starting upload Aug 9 00:53:50.823435035 (7h27m45.254515166s) [upload=7803]: Done starting upload Aug 9 00:53:50.823486734 (7h27m45.254566855s) [upload=7803]: Uploading segments... Aug 9 00:53:50.823519837 (7h27m45.254599968s) [upload=7803]: Got next segment. Inline: true Aug 9 00:53:50.823534765 (7h27m45.254614896s) [upload=7803]: Waiting for error group managing segments... Aug 9 00:53:50.823567247 (7h27m45.254647378s) [upload=7803]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:53:50.823571045 (7h27m45.254651176s) [upload=7803]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:53:50.823573500 (7h27m45.254653621s) [upload=7803]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:53:51.036378597 (7h27m45.467458728s) [upload=7803]: Done uploading segments... Aug 9 00:53:51.036397072 (7h27m45.467477203s) [upload=7803]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.894281807s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 2.492030477s FAIL (2025-08-09 00:55:36 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.492059462s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [..X.X..........................X.X.....X................X...] duration: 1.961324009s FAIL (2025-08-09 00:55:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:53:50.823389357 (7h27m45.254469488s) [upload=7803]: Starting upload Aug 9 00:53:50.823435035 (7h27m45.254515166s) [upload=7803]: Done starting upload Aug 9 00:53:50.823486734 (7h27m45.254566855s) [upload=7803]: Uploading segments... Aug 9 00:53:50.823519837 (7h27m45.254599968s) [upload=7803]: Got next segment. Inline: true Aug 9 00:53:50.823534765 (7h27m45.254614896s) [upload=7803]: Waiting for error group managing segments... Aug 9 00:53:50.823567247 (7h27m45.254647378s) [upload=7803]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:53:50.823571045 (7h27m45.254651176s) [upload=7803]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:53:50.823573500 (7h27m45.254653621s) [upload=7803]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:53:51.036378597 (7h27m45.467458728s) [upload=7803]: Done uploading segments... Aug 9 00:53:51.036397072 (7h27m45.467477203s) [upload=7803]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.961347884s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X.......................X...X.......] duration: 6.085287984s FAIL (2025-08-09 00:55:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 00:48:46.017517689 (7h22m40.44859781s) [upload=7718]: Starting upload Aug 9 00:48:46.017561742 (7h22m40.448641863s) [upload=7718]: Done starting upload Aug 9 00:48:46.017629412 (7h22m40.448709543s) [upload=7718]: Uploading segments... Aug 9 00:48:46.017700498 (7h22m40.448780629s) [upload=7718]: Got next segment. Inline: false Aug 9 00:48:46.017739152 (7h22m40.448819273s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:48:46.017743611 (7h22m40.448823742s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 00:48:46.487633950 (7h22m40.918714081s) [seg_pos={0 0}, upload=7718]: Begin upload segment... Aug 9 00:48:46.487687452 (7h22m40.918767583s) [seg_pos={0 0}, upload=7718]: Done begin upload segment. Aug 9 00:48:46.487711057 (7h22m40.918791188s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.487756925 (7h22m40.918837056s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488097557 (7h22m40.919177688s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488233727 (7h22m40.919313858s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488374266 (7h22m40.919454397s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488628011 (7h22m40.919708142s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488856018 (7h22m40.919936139s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489126416 (7h22m40.920206537s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489348511 (7h22m40.920428642s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489581706 (7h22m40.920661837s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489787100 (7h22m40.920867231s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489998043 (7h22m40.921078174s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490181434 (7h22m40.921261565s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490363022 (7h22m40.921443153s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490552123 (7h22m40.921632254s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490745964 (7h22m40.921826085s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490924466 (7h22m40.922004597s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.491108248 (7h22m40.922188379s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:47.595108697 (7h22m42.026188828s) [upload=7718]: Next returned nil segment Aug 9 00:48:47.595125159 (7h22m42.02620529s) [upload=7718]: Waiting for error group managing segments... Aug 9 00:48:47.877294483 (7h22m42.308374614s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.948937908 (7h22m42.380018039s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.968047280 (7h22m42.399127411s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.520661895 (7h22m42.951742026s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.523938645 (7h22m42.955018776s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.544420141 (7h22m42.975500272s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.566672848 (7h22m42.997752979s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.582416790 (7h22m43.013496921s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.585069256 (7h22m43.016149387s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.658756792 (7h22m43.089836923s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.659623009 (7h22m43.09070314s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.873462627 (7h22m43.304542758s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.259262096 (7h22m43.690342227s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319042805 (7h22m43.750122936s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319066630 (7h22m43.750146761s) [seg_pos={0 0}, upload=7718]: Segment reached optimal threshold of 14 pieces. Aug 9 00:48:49.319407803 (7h22m43.750487934s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: rpc: dial tcp 104.36.82.44:4894: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319452639 (7h22m43.75053277s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UpWmMhEmk5veKZvzsQ3MCxCMnfEKtFgCugBTsYfy9Ejm7G4xD): protocol: read tcp 188.166.225.170:35022->104.36.82.15:4292: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319470654 (7h22m43.750550785s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 9 00:48:49.319573690 (7h22m43.750653821s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b): protocol: write tcp 188.166.225.170:41310->104.36.82.19:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319629567 (7h22m43.750709698s) [seg_pos={0 0}, upload=7718]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 00:48:49.319709189 (7h22m43.75078932s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 00:48:49.319714160 (7h22m43.750794291s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:48:49.635176021 (7h22m44.066256152s) [upload=7718]: Done uploading segments... Aug 9 00:48:49.635194487 (7h22m44.066274618s) [upload=7718]: Upload finished. err: <nil> Duration: 6.085316678s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 2.361580558s FAIL (2025-08-09 00:55:04 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.361609432s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 717.436107ms FAIL (2025-08-09 00:55:04 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 717.456795ms edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [...X.X..........................X.X.....X................X..] duration: 1.854091241s FAIL (2025-08-09 00:54:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:53:50.823389357 (7h27m45.254469488s) [upload=7803]: Starting upload Aug 9 00:53:50.823435035 (7h27m45.254515166s) [upload=7803]: Done starting upload Aug 9 00:53:50.823486734 (7h27m45.254566855s) [upload=7803]: Uploading segments... Aug 9 00:53:50.823519837 (7h27m45.254599968s) [upload=7803]: Got next segment. Inline: true Aug 9 00:53:50.823534765 (7h27m45.254614896s) [upload=7803]: Waiting for error group managing segments... Aug 9 00:53:50.823567247 (7h27m45.254647378s) [upload=7803]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:53:50.823571045 (7h27m45.254651176s) [upload=7803]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:53:50.823573500 (7h27m45.254653621s) [upload=7803]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:53:51.036378597 (7h27m45.467458728s) [upload=7803]: Done uploading segments... Aug 9 00:53:51.036397072 (7h27m45.467477203s) [upload=7803]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.854121108s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 2.214783061s FAIL (2025-08-09 00:54:41 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.214801557s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [....X.X..........................X.X.....X................X.] duration: 1.964555911s FAIL (2025-08-09 00:54:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:53:50.823389357 (7h27m45.254469488s) [upload=7803]: Starting upload Aug 9 00:53:50.823435035 (7h27m45.254515166s) [upload=7803]: Done starting upload Aug 9 00:53:50.823486734 (7h27m45.254566855s) [upload=7803]: Uploading segments... Aug 9 00:53:50.823519837 (7h27m45.254599968s) [upload=7803]: Got next segment. Inline: true Aug 9 00:53:50.823534765 (7h27m45.254614896s) [upload=7803]: Waiting for error group managing segments... Aug 9 00:53:50.823567247 (7h27m45.254647378s) [upload=7803]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:53:50.823571045 (7h27m45.254651176s) [upload=7803]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:53:50.823573500 (7h27m45.254653621s) [upload=7803]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:53:51.036378597 (7h27m45.467458728s) [upload=7803]: Done uploading segments... Aug 9 00:53:51.036397072 (7h27m45.467477203s) [upload=7803]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.964579607s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.254751149s FAIL (2025-08-09 00:54:21 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.254775836s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X.......................X...X......] duration: 6.037137689s FAIL (2025-08-09 00:54:08 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 00:48:46.017517689 (7h22m40.44859781s) [upload=7718]: Starting upload Aug 9 00:48:46.017561742 (7h22m40.448641863s) [upload=7718]: Done starting upload Aug 9 00:48:46.017629412 (7h22m40.448709543s) [upload=7718]: Uploading segments... Aug 9 00:48:46.017700498 (7h22m40.448780629s) [upload=7718]: Got next segment. Inline: false Aug 9 00:48:46.017739152 (7h22m40.448819273s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:48:46.017743611 (7h22m40.448823742s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 00:48:46.487633950 (7h22m40.918714081s) [seg_pos={0 0}, upload=7718]: Begin upload segment... Aug 9 00:48:46.487687452 (7h22m40.918767583s) [seg_pos={0 0}, upload=7718]: Done begin upload segment. Aug 9 00:48:46.487711057 (7h22m40.918791188s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.487756925 (7h22m40.918837056s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488097557 (7h22m40.919177688s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488233727 (7h22m40.919313858s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488374266 (7h22m40.919454397s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488628011 (7h22m40.919708142s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488856018 (7h22m40.919936139s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489126416 (7h22m40.920206537s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489348511 (7h22m40.920428642s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489581706 (7h22m40.920661837s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489787100 (7h22m40.920867231s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489998043 (7h22m40.921078174s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490181434 (7h22m40.921261565s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490363022 (7h22m40.921443153s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490552123 (7h22m40.921632254s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490745964 (7h22m40.921826085s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490924466 (7h22m40.922004597s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.491108248 (7h22m40.922188379s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:47.595108697 (7h22m42.026188828s) [upload=7718]: Next returned nil segment Aug 9 00:48:47.595125159 (7h22m42.02620529s) [upload=7718]: Waiting for error group managing segments... Aug 9 00:48:47.877294483 (7h22m42.308374614s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.948937908 (7h22m42.380018039s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.968047280 (7h22m42.399127411s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.520661895 (7h22m42.951742026s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.523938645 (7h22m42.955018776s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.544420141 (7h22m42.975500272s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.566672848 (7h22m42.997752979s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.582416790 (7h22m43.013496921s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.585069256 (7h22m43.016149387s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.658756792 (7h22m43.089836923s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.659623009 (7h22m43.09070314s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.873462627 (7h22m43.304542758s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.259262096 (7h22m43.690342227s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319042805 (7h22m43.750122936s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319066630 (7h22m43.750146761s) [seg_pos={0 0}, upload=7718]: Segment reached optimal threshold of 14 pieces. Aug 9 00:48:49.319407803 (7h22m43.750487934s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: rpc: dial tcp 104.36.82.44:4894: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319452639 (7h22m43.75053277s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UpWmMhEmk5veKZvzsQ3MCxCMnfEKtFgCugBTsYfy9Ejm7G4xD): protocol: read tcp 188.166.225.170:35022->104.36.82.15:4292: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319470654 (7h22m43.750550785s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 9 00:48:49.319573690 (7h22m43.750653821s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b): protocol: write tcp 188.166.225.170:41310->104.36.82.19:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319629567 (7h22m43.750709698s) [seg_pos={0 0}, upload=7718]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 00:48:49.319709189 (7h22m43.75078932s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 00:48:49.319714160 (7h22m43.750794291s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:48:49.635176021 (7h22m44.066256152s) [upload=7718]: Done uploading segments... Aug 9 00:48:49.635194487 (7h22m44.066274618s) [upload=7718]: Upload finished. err: <nil> Duration: 6.037163839s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 753.814816ms FAIL (2025-08-09 00:53:53 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 753.848271ms edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [.....X.X..........................X.X.....X................X] duration: 856.248635ms FAIL (2025-08-09 00:53:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:53:50.823389357 (7h27m45.254469488s) [upload=7803]: Starting upload Aug 9 00:53:50.823435035 (7h27m45.254515166s) [upload=7803]: Done starting upload Aug 9 00:53:50.823486734 (7h27m45.254566855s) [upload=7803]: Uploading segments... Aug 9 00:53:50.823519837 (7h27m45.254599968s) [upload=7803]: Got next segment. Inline: true Aug 9 00:53:50.823534765 (7h27m45.254614896s) [upload=7803]: Waiting for error group managing segments... Aug 9 00:53:50.823567247 (7h27m45.254647378s) [upload=7803]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:53:50.823571045 (7h27m45.254651176s) [upload=7803]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:53:50.823573500 (7h27m45.254653621s) [upload=7803]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:53:51.036378597 (7h27m45.467458728s) [upload=7803]: Done uploading segments... Aug 9 00:53:51.036397072 (7h27m45.467477203s) [upload=7803]: Upload finished. err: metaclient: Too Many Requests Duration:* 856.269073ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.353732651s FAIL (2025-08-09 00:53:41 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.353753223s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [X.....X.X..........................X.X.....X................] duration: 2.09785904s FAIL (2025-08-09 00:53:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:41:49.183606846 (7h15m43.614686977s) [upload=7605]: Starting upload Aug 9 00:41:49.183643877 (7h15m43.614723998s) [upload=7605]: Done starting upload Aug 9 00:41:49.183694574 (7h15m43.614774705s) [upload=7605]: Uploading segments... Aug 9 00:41:49.183722818 (7h15m43.614802949s) [upload=7605]: Got next segment. Inline: true Aug 9 00:41:49.183726575 (7h15m43.614806696s) [upload=7605]: Waiting for error group managing segments... Aug 9 00:41:49.183755280 (7h15m43.614835411s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:41:49.183758957 (7h15m43.614839088s) [upload=7605]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:41:49.183761503 (7h15m43.614841634s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:41:49.397497653 (7h15m43.828577784s) [upload=7605]: Done uploading segments... Aug 9 00:41:49.397515988 (7h15m43.828596119s) [upload=7605]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.097878879s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X.......................X...X.....] duration: 6.051968664s FAIL (2025-08-09 00:53:18 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 00:48:46.017517689 (7h22m40.44859781s) [upload=7718]: Starting upload Aug 9 00:48:46.017561742 (7h22m40.448641863s) [upload=7718]: Done starting upload Aug 9 00:48:46.017629412 (7h22m40.448709543s) [upload=7718]: Uploading segments... Aug 9 00:48:46.017700498 (7h22m40.448780629s) [upload=7718]: Got next segment. Inline: false Aug 9 00:48:46.017739152 (7h22m40.448819273s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:48:46.017743611 (7h22m40.448823742s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 00:48:46.487633950 (7h22m40.918714081s) [seg_pos={0 0}, upload=7718]: Begin upload segment... Aug 9 00:48:46.487687452 (7h22m40.918767583s) [seg_pos={0 0}, upload=7718]: Done begin upload segment. Aug 9 00:48:46.487711057 (7h22m40.918791188s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.487756925 (7h22m40.918837056s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488097557 (7h22m40.919177688s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488233727 (7h22m40.919313858s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488374266 (7h22m40.919454397s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488628011 (7h22m40.919708142s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488856018 (7h22m40.919936139s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489126416 (7h22m40.920206537s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489348511 (7h22m40.920428642s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489581706 (7h22m40.920661837s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489787100 (7h22m40.920867231s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489998043 (7h22m40.921078174s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490181434 (7h22m40.921261565s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490363022 (7h22m40.921443153s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490552123 (7h22m40.921632254s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490745964 (7h22m40.921826085s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490924466 (7h22m40.922004597s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.491108248 (7h22m40.922188379s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:47.595108697 (7h22m42.026188828s) [upload=7718]: Next returned nil segment Aug 9 00:48:47.595125159 (7h22m42.02620529s) [upload=7718]: Waiting for error group managing segments... Aug 9 00:48:47.877294483 (7h22m42.308374614s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.948937908 (7h22m42.380018039s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.968047280 (7h22m42.399127411s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.520661895 (7h22m42.951742026s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.523938645 (7h22m42.955018776s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.544420141 (7h22m42.975500272s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.566672848 (7h22m42.997752979s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.582416790 (7h22m43.013496921s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.585069256 (7h22m43.016149387s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.658756792 (7h22m43.089836923s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.659623009 (7h22m43.09070314s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.873462627 (7h22m43.304542758s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.259262096 (7h22m43.690342227s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319042805 (7h22m43.750122936s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319066630 (7h22m43.750146761s) [seg_pos={0 0}, upload=7718]: Segment reached optimal threshold of 14 pieces. Aug 9 00:48:49.319407803 (7h22m43.750487934s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: rpc: dial tcp 104.36.82.44:4894: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319452639 (7h22m43.75053277s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UpWmMhEmk5veKZvzsQ3MCxCMnfEKtFgCugBTsYfy9Ejm7G4xD): protocol: read tcp 188.166.225.170:35022->104.36.82.15:4292: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319470654 (7h22m43.750550785s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 9 00:48:49.319573690 (7h22m43.750653821s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b): protocol: write tcp 188.166.225.170:41310->104.36.82.19:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319629567 (7h22m43.750709698s) [seg_pos={0 0}, upload=7718]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 00:48:49.319709189 (7h22m43.75078932s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 00:48:49.319714160 (7h22m43.750794291s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:48:49.635176021 (7h22m44.066256152s) [upload=7718]: Done uploading segments... Aug 9 00:48:49.635194487 (7h22m44.066274618s) [upload=7718]: Upload finished. err: <nil> Duration: 6.051988252s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [.X.....X.X..........................X.X.....X...............] duration: 1.979104692s FAIL (2025-08-09 00:53:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:41:49.183606846 (7h15m43.614686977s) [upload=7605]: Starting upload Aug 9 00:41:49.183643877 (7h15m43.614723998s) [upload=7605]: Done starting upload Aug 9 00:41:49.183694574 (7h15m43.614774705s) [upload=7605]: Uploading segments... Aug 9 00:41:49.183722818 (7h15m43.614802949s) [upload=7605]: Got next segment. Inline: true Aug 9 00:41:49.183726575 (7h15m43.614806696s) [upload=7605]: Waiting for error group managing segments... Aug 9 00:41:49.183755280 (7h15m43.614835411s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:41:49.183758957 (7h15m43.614839088s) [upload=7605]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:41:49.183761503 (7h15m43.614841634s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:41:49.397497653 (7h15m43.828577784s) [upload=7605]: Done uploading segments... Aug 9 00:41:49.397515988 (7h15m43.828596119s) [upload=7605]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.979144297s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 737.770372ms FAIL (2025-08-09 00:52:56 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 737.79481ms edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [..X.....X.X..........................X.X.....X..............] duration: 2.281686739s FAIL (2025-08-09 00:52:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:41:49.183606846 (7h15m43.614686977s) [upload=7605]: Starting upload Aug 9 00:41:49.183643877 (7h15m43.614723998s) [upload=7605]: Done starting upload Aug 9 00:41:49.183694574 (7h15m43.614774705s) [upload=7605]: Uploading segments... Aug 9 00:41:49.183722818 (7h15m43.614802949s) [upload=7605]: Got next segment. Inline: true Aug 9 00:41:49.183726575 (7h15m43.614806696s) [upload=7605]: Waiting for error group managing segments... Aug 9 00:41:49.183755280 (7h15m43.614835411s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:41:49.183758957 (7h15m43.614839088s) [upload=7605]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:41:49.183761503 (7h15m43.614841634s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:41:49.397497653 (7h15m43.828577784s) [upload=7605]: Done uploading segments... Aug 9 00:41:49.397515988 (7h15m43.828596119s) [upload=7605]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.281733858s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.317211306s FAIL (2025-08-09 00:52:30 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.317233388s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X.......................X...X....] duration: 6.470258414s FAIL (2025-08-09 00:52:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 00:48:46.017517689 (7h22m40.44859781s) [upload=7718]: Starting upload Aug 9 00:48:46.017561742 (7h22m40.448641863s) [upload=7718]: Done starting upload Aug 9 00:48:46.017629412 (7h22m40.448709543s) [upload=7718]: Uploading segments... Aug 9 00:48:46.017700498 (7h22m40.448780629s) [upload=7718]: Got next segment. Inline: false Aug 9 00:48:46.017739152 (7h22m40.448819273s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:48:46.017743611 (7h22m40.448823742s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 00:48:46.487633950 (7h22m40.918714081s) [seg_pos={0 0}, upload=7718]: Begin upload segment... Aug 9 00:48:46.487687452 (7h22m40.918767583s) [seg_pos={0 0}, upload=7718]: Done begin upload segment. Aug 9 00:48:46.487711057 (7h22m40.918791188s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.487756925 (7h22m40.918837056s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488097557 (7h22m40.919177688s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488233727 (7h22m40.919313858s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488374266 (7h22m40.919454397s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488628011 (7h22m40.919708142s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488856018 (7h22m40.919936139s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489126416 (7h22m40.920206537s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489348511 (7h22m40.920428642s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489581706 (7h22m40.920661837s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489787100 (7h22m40.920867231s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489998043 (7h22m40.921078174s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490181434 (7h22m40.921261565s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490363022 (7h22m40.921443153s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490552123 (7h22m40.921632254s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490745964 (7h22m40.921826085s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490924466 (7h22m40.922004597s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.491108248 (7h22m40.922188379s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:47.595108697 (7h22m42.026188828s) [upload=7718]: Next returned nil segment Aug 9 00:48:47.595125159 (7h22m42.02620529s) [upload=7718]: Waiting for error group managing segments... Aug 9 00:48:47.877294483 (7h22m42.308374614s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.948937908 (7h22m42.380018039s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.968047280 (7h22m42.399127411s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.520661895 (7h22m42.951742026s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.523938645 (7h22m42.955018776s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.544420141 (7h22m42.975500272s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.566672848 (7h22m42.997752979s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.582416790 (7h22m43.013496921s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.585069256 (7h22m43.016149387s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.658756792 (7h22m43.089836923s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.659623009 (7h22m43.09070314s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.873462627 (7h22m43.304542758s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.259262096 (7h22m43.690342227s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319042805 (7h22m43.750122936s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319066630 (7h22m43.750146761s) [seg_pos={0 0}, upload=7718]: Segment reached optimal threshold of 14 pieces. Aug 9 00:48:49.319407803 (7h22m43.750487934s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: rpc: dial tcp 104.36.82.44:4894: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319452639 (7h22m43.75053277s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UpWmMhEmk5veKZvzsQ3MCxCMnfEKtFgCugBTsYfy9Ejm7G4xD): protocol: read tcp 188.166.225.170:35022->104.36.82.15:4292: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319470654 (7h22m43.750550785s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 9 00:48:49.319573690 (7h22m43.750653821s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b): protocol: write tcp 188.166.225.170:41310->104.36.82.19:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319629567 (7h22m43.750709698s) [seg_pos={0 0}, upload=7718]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 00:48:49.319709189 (7h22m43.75078932s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 00:48:49.319714160 (7h22m43.750794291s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:48:49.635176021 (7h22m44.066256152s) [upload=7718]: Done uploading segments... Aug 9 00:48:49.635194487 (7h22m44.066274618s) [upload=7718]: Upload finished. err: <nil> Duration: 6.470286267s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 725.877059ms FAIL (2025-08-09 00:52:20 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 725.926462ms edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [...X.....X.X..........................X.X.....X.............] duration: 1.822121018s FAIL (2025-08-09 00:52:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:41:49.183606846 (7h15m43.614686977s) [upload=7605]: Starting upload Aug 9 00:41:49.183643877 (7h15m43.614723998s) [upload=7605]: Done starting upload Aug 9 00:41:49.183694574 (7h15m43.614774705s) [upload=7605]: Uploading segments... Aug 9 00:41:49.183722818 (7h15m43.614802949s) [upload=7605]: Got next segment. Inline: true Aug 9 00:41:49.183726575 (7h15m43.614806696s) [upload=7605]: Waiting for error group managing segments... Aug 9 00:41:49.183755280 (7h15m43.614835411s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:41:49.183758957 (7h15m43.614839088s) [upload=7605]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:41:49.183761503 (7h15m43.614841634s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:41:49.397497653 (7h15m43.828577784s) [upload=7605]: Done uploading segments... Aug 9 00:41:49.397515988 (7h15m43.828596119s) [upload=7605]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.822146918s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.259869321s FAIL (2025-08-09 00:52:04 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.25989018s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 763.160252ms FAIL (2025-08-09 00:51:56 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 763.18561ms edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X.......................X...X...] duration: 6.449481289s FAIL (2025-08-09 00:51:09 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 00:48:46.017517689 (7h22m40.44859781s) [upload=7718]: Starting upload Aug 9 00:48:46.017561742 (7h22m40.448641863s) [upload=7718]: Done starting upload Aug 9 00:48:46.017629412 (7h22m40.448709543s) [upload=7718]: Uploading segments... Aug 9 00:48:46.017700498 (7h22m40.448780629s) [upload=7718]: Got next segment. Inline: false Aug 9 00:48:46.017739152 (7h22m40.448819273s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:48:46.017743611 (7h22m40.448823742s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 00:48:46.487633950 (7h22m40.918714081s) [seg_pos={0 0}, upload=7718]: Begin upload segment... Aug 9 00:48:46.487687452 (7h22m40.918767583s) [seg_pos={0 0}, upload=7718]: Done begin upload segment. Aug 9 00:48:46.487711057 (7h22m40.918791188s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.487756925 (7h22m40.918837056s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488097557 (7h22m40.919177688s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488233727 (7h22m40.919313858s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488374266 (7h22m40.919454397s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488628011 (7h22m40.919708142s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488856018 (7h22m40.919936139s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489126416 (7h22m40.920206537s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489348511 (7h22m40.920428642s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489581706 (7h22m40.920661837s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489787100 (7h22m40.920867231s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489998043 (7h22m40.921078174s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490181434 (7h22m40.921261565s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490363022 (7h22m40.921443153s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490552123 (7h22m40.921632254s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490745964 (7h22m40.921826085s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490924466 (7h22m40.922004597s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.491108248 (7h22m40.922188379s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:47.595108697 (7h22m42.026188828s) [upload=7718]: Next returned nil segment Aug 9 00:48:47.595125159 (7h22m42.02620529s) [upload=7718]: Waiting for error group managing segments... Aug 9 00:48:47.877294483 (7h22m42.308374614s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.948937908 (7h22m42.380018039s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.968047280 (7h22m42.399127411s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.520661895 (7h22m42.951742026s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.523938645 (7h22m42.955018776s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.544420141 (7h22m42.975500272s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.566672848 (7h22m42.997752979s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.582416790 (7h22m43.013496921s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.585069256 (7h22m43.016149387s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.658756792 (7h22m43.089836923s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.659623009 (7h22m43.09070314s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.873462627 (7h22m43.304542758s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.259262096 (7h22m43.690342227s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319042805 (7h22m43.750122936s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319066630 (7h22m43.750146761s) [seg_pos={0 0}, upload=7718]: Segment reached optimal threshold of 14 pieces. Aug 9 00:48:49.319407803 (7h22m43.750487934s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: rpc: dial tcp 104.36.82.44:4894: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319452639 (7h22m43.75053277s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UpWmMhEmk5veKZvzsQ3MCxCMnfEKtFgCugBTsYfy9Ejm7G4xD): protocol: read tcp 188.166.225.170:35022->104.36.82.15:4292: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319470654 (7h22m43.750550785s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 9 00:48:49.319573690 (7h22m43.750653821s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b): protocol: write tcp 188.166.225.170:41310->104.36.82.19:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319629567 (7h22m43.750709698s) [seg_pos={0 0}, upload=7718]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 00:48:49.319709189 (7h22m43.75078932s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 00:48:49.319714160 (7h22m43.750794291s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:48:49.635176021 (7h22m44.066256152s) [upload=7718]: Done uploading segments... Aug 9 00:48:49.635194487 (7h22m44.066274618s) [upload=7718]: Upload finished. err: <nil> Duration: 6.449508291s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [....X.....X.X..........................X.X.....X............] duration: 1.974983109s FAIL (2025-08-09 00:51:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:41:49.183606846 (7h15m43.614686977s) [upload=7605]: Starting upload Aug 9 00:41:49.183643877 (7h15m43.614723998s) [upload=7605]: Done starting upload Aug 9 00:41:49.183694574 (7h15m43.614774705s) [upload=7605]: Uploading segments... Aug 9 00:41:49.183722818 (7h15m43.614802949s) [upload=7605]: Got next segment. Inline: true Aug 9 00:41:49.183726575 (7h15m43.614806696s) [upload=7605]: Waiting for error group managing segments... Aug 9 00:41:49.183755280 (7h15m43.614835411s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:41:49.183758957 (7h15m43.614839088s) [upload=7605]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:41:49.183761503 (7h15m43.614841634s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:41:49.397497653 (7h15m43.828577784s) [upload=7605]: Done uploading segments... Aug 9 00:41:49.397515988 (7h15m43.828596119s) [upload=7605]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.975007776s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 2.346508377s FAIL (2025-08-09 00:51:07 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.346530478s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 800.340789ms FAIL (2025-08-09 00:50:54 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 800.365826ms edge authservice http us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 731.977338ms FAIL (2025-08-09 00:50:17 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 732.000883ms edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 7 out of 60 is failed (threshold=3) [X....X.....X.X..........................X.X.....X...........] duration: 1.916220216s FAIL (2025-08-09 00:50:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:41:49.183606846 (7h15m43.614686977s) [upload=7605]: Starting upload Aug 9 00:41:49.183643877 (7h15m43.614723998s) [upload=7605]: Done starting upload Aug 9 00:41:49.183694574 (7h15m43.614774705s) [upload=7605]: Uploading segments... Aug 9 00:41:49.183722818 (7h15m43.614802949s) [upload=7605]: Got next segment. Inline: true Aug 9 00:41:49.183726575 (7h15m43.614806696s) [upload=7605]: Waiting for error group managing segments... Aug 9 00:41:49.183755280 (7h15m43.614835411s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:41:49.183758957 (7h15m43.614839088s) [upload=7605]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:41:49.183761503 (7h15m43.614841634s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:41:49.397497653 (7h15m43.828577784s) [upload=7605]: Done uploading segments... Aug 9 00:41:49.397515988 (7h15m43.828596119s) [upload=7605]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.916255434s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................X.......................X...X..] duration: 8.095176279s FAIL (2025-08-09 00:50:00 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 00:48:46.017517689 (7h22m40.44859781s) [upload=7718]: Starting upload Aug 9 00:48:46.017561742 (7h22m40.448641863s) [upload=7718]: Done starting upload Aug 9 00:48:46.017629412 (7h22m40.448709543s) [upload=7718]: Uploading segments... Aug 9 00:48:46.017700498 (7h22m40.448780629s) [upload=7718]: Got next segment. Inline: false Aug 9 00:48:46.017739152 (7h22m40.448819273s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:48:46.017743611 (7h22m40.448823742s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 00:48:46.487633950 (7h22m40.918714081s) [seg_pos={0 0}, upload=7718]: Begin upload segment... Aug 9 00:48:46.487687452 (7h22m40.918767583s) [seg_pos={0 0}, upload=7718]: Done begin upload segment. Aug 9 00:48:46.487711057 (7h22m40.918791188s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.487756925 (7h22m40.918837056s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488097557 (7h22m40.919177688s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488233727 (7h22m40.919313858s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488374266 (7h22m40.919454397s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488628011 (7h22m40.919708142s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488856018 (7h22m40.919936139s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489126416 (7h22m40.920206537s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489348511 (7h22m40.920428642s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489581706 (7h22m40.920661837s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489787100 (7h22m40.920867231s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489998043 (7h22m40.921078174s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490181434 (7h22m40.921261565s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490363022 (7h22m40.921443153s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490552123 (7h22m40.921632254s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490745964 (7h22m40.921826085s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490924466 (7h22m40.922004597s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.491108248 (7h22m40.922188379s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:47.595108697 (7h22m42.026188828s) [upload=7718]: Next returned nil segment Aug 9 00:48:47.595125159 (7h22m42.02620529s) [upload=7718]: Waiting for error group managing segments... Aug 9 00:48:47.877294483 (7h22m42.308374614s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.948937908 (7h22m42.380018039s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.968047280 (7h22m42.399127411s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.520661895 (7h22m42.951742026s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.523938645 (7h22m42.955018776s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.544420141 (7h22m42.975500272s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.566672848 (7h22m42.997752979s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.582416790 (7h22m43.013496921s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.585069256 (7h22m43.016149387s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.658756792 (7h22m43.089836923s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.659623009 (7h22m43.09070314s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.873462627 (7h22m43.304542758s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.259262096 (7h22m43.690342227s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319042805 (7h22m43.750122936s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319066630 (7h22m43.750146761s) [seg_pos={0 0}, upload=7718]: Segment reached optimal threshold of 14 pieces. Aug 9 00:48:49.319407803 (7h22m43.750487934s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: rpc: dial tcp 104.36.82.44:4894: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319452639 (7h22m43.75053277s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UpWmMhEmk5veKZvzsQ3MCxCMnfEKtFgCugBTsYfy9Ejm7G4xD): protocol: read tcp 188.166.225.170:35022->104.36.82.15:4292: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319470654 (7h22m43.750550785s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 9 00:48:49.319573690 (7h22m43.750653821s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b): protocol: write tcp 188.166.225.170:41310->104.36.82.19:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319629567 (7h22m43.750709698s) [seg_pos={0 0}, upload=7718]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 00:48:49.319709189 (7h22m43.75078932s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 00:48:49.319714160 (7h22m43.750794291s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:48:49.635176021 (7h22m44.066256152s) [upload=7718]: Done uploading segments... Aug 9 00:48:49.635194487 (7h22m44.066274618s) [upload=7718]: Upload finished. err: <nil> Duration: 8.095271872s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 2.281964153s FAIL (2025-08-09 00:49:59 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.281988089s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 7 out of 60 is failed (threshold=3) [.X....X.....X.X..........................X.X.....X..........] duration: 2.106711671s FAIL (2025-08-09 00:49:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:41:49.183606846 (7h15m43.614686977s) [upload=7605]: Starting upload Aug 9 00:41:49.183643877 (7h15m43.614723998s) [upload=7605]: Done starting upload Aug 9 00:41:49.183694574 (7h15m43.614774705s) [upload=7605]: Uploading segments... Aug 9 00:41:49.183722818 (7h15m43.614802949s) [upload=7605]: Got next segment. Inline: true Aug 9 00:41:49.183726575 (7h15m43.614806696s) [upload=7605]: Waiting for error group managing segments... Aug 9 00:41:49.183755280 (7h15m43.614835411s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:41:49.183758957 (7h15m43.614839088s) [upload=7605]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:41:49.183761503 (7h15m43.614841634s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:41:49.397497653 (7h15m43.828577784s) [upload=7605]: Done uploading segments... Aug 9 00:41:49.397515988 (7h15m43.828596119s) [upload=7605]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.106740696s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 746.274152ms FAIL (2025-08-09 00:49:21 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 746.298519ms edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 2.31827965s FAIL (2025-08-09 00:49:18 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.318294539s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................X.......................X...X.] duration: 6.641473265s FAIL (2025-08-09 00:49:17 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 00:48:46.017517689 (7h22m40.44859781s) [upload=7718]: Starting upload Aug 9 00:48:46.017561742 (7h22m40.448641863s) [upload=7718]: Done starting upload Aug 9 00:48:46.017629412 (7h22m40.448709543s) [upload=7718]: Uploading segments... Aug 9 00:48:46.017700498 (7h22m40.448780629s) [upload=7718]: Got next segment. Inline: false Aug 9 00:48:46.017739152 (7h22m40.448819273s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:48:46.017743611 (7h22m40.448823742s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 00:48:46.487633950 (7h22m40.918714081s) [seg_pos={0 0}, upload=7718]: Begin upload segment... Aug 9 00:48:46.487687452 (7h22m40.918767583s) [seg_pos={0 0}, upload=7718]: Done begin upload segment. Aug 9 00:48:46.487711057 (7h22m40.918791188s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.487756925 (7h22m40.918837056s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488097557 (7h22m40.919177688s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488233727 (7h22m40.919313858s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488374266 (7h22m40.919454397s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488628011 (7h22m40.919708142s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488856018 (7h22m40.919936139s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489126416 (7h22m40.920206537s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489348511 (7h22m40.920428642s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489581706 (7h22m40.920661837s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489787100 (7h22m40.920867231s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489998043 (7h22m40.921078174s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490181434 (7h22m40.921261565s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490363022 (7h22m40.921443153s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490552123 (7h22m40.921632254s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490745964 (7h22m40.921826085s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490924466 (7h22m40.922004597s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.491108248 (7h22m40.922188379s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:47.595108697 (7h22m42.026188828s) [upload=7718]: Next returned nil segment Aug 9 00:48:47.595125159 (7h22m42.02620529s) [upload=7718]: Waiting for error group managing segments... Aug 9 00:48:47.877294483 (7h22m42.308374614s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.948937908 (7h22m42.380018039s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.968047280 (7h22m42.399127411s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.520661895 (7h22m42.951742026s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.523938645 (7h22m42.955018776s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.544420141 (7h22m42.975500272s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.566672848 (7h22m42.997752979s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.582416790 (7h22m43.013496921s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.585069256 (7h22m43.016149387s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.658756792 (7h22m43.089836923s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.659623009 (7h22m43.09070314s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.873462627 (7h22m43.304542758s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.259262096 (7h22m43.690342227s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319042805 (7h22m43.750122936s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319066630 (7h22m43.750146761s) [seg_pos={0 0}, upload=7718]: Segment reached optimal threshold of 14 pieces. Aug 9 00:48:49.319407803 (7h22m43.750487934s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: rpc: dial tcp 104.36.82.44:4894: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319452639 (7h22m43.75053277s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UpWmMhEmk5veKZvzsQ3MCxCMnfEKtFgCugBTsYfy9Ejm7G4xD): protocol: read tcp 188.166.225.170:35022->104.36.82.15:4292: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319470654 (7h22m43.750550785s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 9 00:48:49.319573690 (7h22m43.750653821s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b): protocol: write tcp 188.166.225.170:41310->104.36.82.19:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319629567 (7h22m43.750709698s) [seg_pos={0 0}, upload=7718]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 00:48:49.319709189 (7h22m43.75078932s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 00:48:49.319714160 (7h22m43.750794291s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:48:49.635176021 (7h22m44.066256152s) [upload=7718]: Done uploading segments... Aug 9 00:48:49.635194487 (7h22m44.066274618s) [upload=7718]: Upload finished. err: <nil> Duration: 6.64150169s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 784.417062ms FAIL (2025-08-09 00:48:57 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 784.440497ms edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................X.......................X...X] duration: 4.965525421s FAIL (2025-08-09 00:48:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 9 00:48:46.017517689 (7h22m40.44859781s) [upload=7718]: Starting upload Aug 9 00:48:46.017561742 (7h22m40.448641863s) [upload=7718]: Done starting upload Aug 9 00:48:46.017629412 (7h22m40.448709543s) [upload=7718]: Uploading segments... Aug 9 00:48:46.017700498 (7h22m40.448780629s) [upload=7718]: Got next segment. Inline: false Aug 9 00:48:46.017739152 (7h22m40.448819273s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:48:46.017743611 (7h22m40.448823742s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 00:48:46.487633950 (7h22m40.918714081s) [seg_pos={0 0}, upload=7718]: Begin upload segment... Aug 9 00:48:46.487687452 (7h22m40.918767583s) [seg_pos={0 0}, upload=7718]: Done begin upload segment. Aug 9 00:48:46.487711057 (7h22m40.918791188s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.487756925 (7h22m40.918837056s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488097557 (7h22m40.919177688s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488233727 (7h22m40.919313858s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488374266 (7h22m40.919454397s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488628011 (7h22m40.919708142s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.488856018 (7h22m40.919936139s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489126416 (7h22m40.920206537s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489348511 (7h22m40.920428642s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489581706 (7h22m40.920661837s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489787100 (7h22m40.920867231s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.489998043 (7h22m40.921078174s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490181434 (7h22m40.921261565s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490363022 (7h22m40.921443153s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490552123 (7h22m40.921632254s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490745964 (7h22m40.921826085s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.490924466 (7h22m40.922004597s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:46.491108248 (7h22m40.922188379s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Uploading piece... Aug 9 00:48:47.595108697 (7h22m42.026188828s) [upload=7718]: Next returned nil segment Aug 9 00:48:47.595125159 (7h22m42.02620529s) [upload=7718]: Waiting for error group managing segments... Aug 9 00:48:47.877294483 (7h22m42.308374614s) [noise=true , address=64.49.13.138:4484 , piece_id=XJCMFVWHATEUVJLLD5VQFYG4ZXDH7TNBDOEFH7KV2JL4HM4X4IHQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.948937908 (7h22m42.380018039s) [noise=true , address=64.49.13.137:4500 , piece_id=74XIQN5DD2QOUR5D7DRT5CUCENQR74UKK45UEPH6LOHJ3HBURHYQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:47.968047280 (7h22m42.399127411s) [noise=true , address=64.49.13.169:4710 , piece_id=ZEKXCRSD434VSC2ORGR4QPLHOHMJCBQYGVTWIJLWR2XFM5DG5AWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.520661895 (7h22m42.951742026s) [noise=true , address=104.36.82.22:4512 , piece_id=ERJOH5YWZ3BAKEMSSWW26EYQBP3ZEDTYMACULERDJ5MBVHGUJP6Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.523938645 (7h22m42.955018776s) [noise=true , address=45.91.116.11:4138 , piece_id=UWX4NN52D37ZJXXA5CKNE4N6ZYEWEYPMPPDZ37QM3B5CMQ2MLGMA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.544420141 (7h22m42.975500272s) [noise=true , address=104.36.82.35:4588 , piece_id=S2JRBW2BJDMVTOWAN66SAIEOFWDW2ZMV6PCC4JSTUAMZWT566YVA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.566672848 (7h22m42.997752979s) [noise=true , address=45.91.116.11:4141 , piece_id=UBHP6JVK2FUFYLRVOULFRBMSOTUJCE56E76BU4EI23FHGF6SXPWQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.582416790 (7h22m43.013496921s) [noise=true , address=104.36.82.60:4370 , piece_id=72LHFQLVXCW5X2EYKC7TUUQICLE7R4KXXOE3UVEBC6K24KUON3LA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.585069256 (7h22m43.016149387s) [noise=true , address=104.36.82.62:4435 , piece_id=QTZA2CCHRZK474D4T7MK5DDY2ZUB25XAOWDYYICFWYXHHN4U4GSA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.658756792 (7h22m43.089836923s) [noise=true , address=104.36.82.37:4663 , piece_id=6PS327IZB4Y26F3W7B6PR5SAVZXJE6OIH5ZFHNOLZJXDI4LICQZA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.659623009 (7h22m43.09070314s) [noise=true , address=104.36.82.14:4236 , piece_id=IVEBYXCNOQ3AQ4TOBTJEJIJKKB7MB4CHNZFQF2I2DWADEIMFTT2A, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:48.873462627 (7h22m43.304542758s) [noise=true , address=104.36.82.4:4812 , piece_id=JX6L3CMRDWCGHMIQI2PIP4LHVXNNHLSKS2RNLAYBMXFJOZU2QCNA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.259262096 (7h22m43.690342227s) [noise=true , address=104.36.82.9:4017 , piece_id=A737253TXYKIQFEZ2CDOR5AQGKFBMONN3MCLIALGZBAQ3IVPENNQ, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319042805 (7h22m43.750122936s) [noise=true , address=104.36.82.45:4909 , piece_id=DRFHX4YPP5SMIBBBUQBGDBSVPPLP3LOJKEU5D3VAP6ZFK5YSGU5Q, seg_pos={0 0}, upload=7718]: Done uploading piece. err: <nil> Aug 9 00:48:49.319066630 (7h22m43.750146761s) [seg_pos={0 0}, upload=7718]: Segment reached optimal threshold of 14 pieces. Aug 9 00:48:49.319407803 (7h22m43.750487934s) [noise=true , address=104.36.82.44:4894 , piece_id=YEAPGAAV4XQQTGBJ7AIA4KJPZFPSQ44JDDCCZ2Q4OOBZWDOATFWA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: rpc: dial tcp 104.36.82.44:4894: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319452639 (7h22m43.75053277s) [noise=true , address=104.36.82.15:4292 , piece_id=ZG53SGOH4M4OECCYW5KQKS6XL5PK3H322GP6LX5WFY7O454RAUTA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UpWmMhEmk5veKZvzsQ3MCxCMnfEKtFgCugBTsYfy9Ejm7G4xD): protocol: read tcp 188.166.225.170:35022->104.36.82.15:4292: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319470654 (7h22m43.750550785s) [noise=true , address=104.36.82.45:4918 , piece_id=TE7ASCIZM6CWQGUNON6L2WETJPGVRAGISW6LAXHPXHTBZPBEXPCA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Nw78dd9WPnmCwv5bFQR3VpV39776Pye2QVGSNMTs3zq2BtaGJ): protocol: expected piece hash; context canceled; EOF Aug 9 00:48:49.319573690 (7h22m43.750653821s) [noise=true , address=104.36.82.19:4404 , piece_id=5VYJZAMM3H22UKBZWAA2SERPBXBOHUXC7HCIO2UNCJEDIXG3XQYA, seg_pos={0 0}, upload=7718]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b): protocol: write tcp 188.166.225.170:41310->104.36.82.19:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 9 00:48:49.319629567 (7h22m43.750709698s) [seg_pos={0 0}, upload=7718]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 00:48:49.319709189 (7h22m43.75078932s) [upload=7718]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 00:48:49.319714160 (7h22m43.750794291s) [upload=7718]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:48:49.635176021 (7h22m44.066256152s) [upload=7718]: Done uploading segments... Aug 9 00:48:49.635194487 (7h22m44.066274618s) [upload=7718]: Upload finished. err: <nil> Duration: 4.96555096s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 756.829341ms FAIL (2025-08-09 00:48:38 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 756.854177ms edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 7 out of 60 is failed (threshold=3) [..X....X.....X.X..........................X.X.....X.........] duration: 1.974684915s FAIL (2025-08-09 00:48:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:41:49.183606846 (7h15m43.614686977s) [upload=7605]: Starting upload Aug 9 00:41:49.183643877 (7h15m43.614723998s) [upload=7605]: Done starting upload Aug 9 00:41:49.183694574 (7h15m43.614774705s) [upload=7605]: Uploading segments... Aug 9 00:41:49.183722818 (7h15m43.614802949s) [upload=7605]: Got next segment. Inline: true Aug 9 00:41:49.183726575 (7h15m43.614806696s) [upload=7605]: Waiting for error group managing segments... Aug 9 00:41:49.183755280 (7h15m43.614835411s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 00:41:49.183758957 (7h15m43.614839088s) [upload=7605]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 00:41:49.183761503 (7h15m43.614841634s) [upload=7605]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 00:41:49.397497653 (7h15m43.828577784s) [upload=7605]: Done uploading segments... Aug 9 00:41:49.397515988 (7h15m43.828596119s) [upload=7605]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.974720925s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 2.291331838s FAIL (2025-08-09 00:48:14 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2.291365481s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 731.924598ms FAIL (2025-08-09 00:48:03 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout Duration: 731.95711ms edge authservice http us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)