Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
5.970874438s
FAIL (2025-08-21 20:03:59 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.970898413s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [....X.X.......X.................X...........................] duration:
5.970891571s
FAIL (2025-08-21 20:03:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.970923401s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
5.795653351s
FAIL (2025-08-21 20:03:13 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.795676575s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [.....X.X.......X.................X..........................] duration:
6.197884746s
FAIL (2025-08-21 20:03:09 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.197914051s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...........X..................X.............................] duration:
1.8935231s
FAIL (2025-08-21 20:02:58 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.893551975s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.736371214s
FAIL (2025-08-21 20:02:44 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.736392826s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [X.....X.X.......X.................X.........................] duration:
6.044800469s
FAIL (2025-08-21 20:02:38 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.044841057s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [............X..................X............................] duration:
2.003336974s
FAIL (2025-08-21 20:02:28 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.003355119s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
5.784155099s
FAIL (2025-08-21 20:02:08 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.784184103s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.X.....X.X.......X.................X........................] duration:
6.150876913s
FAIL (2025-08-21 20:01:58 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.150897653s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
5.812338161s
FAIL (2025-08-21 20:01:44 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.81236875s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.............X..................X...........................] duration:
1.882891959s
FAIL (2025-08-21 20:01:25 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.882955269s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [X.X.....X.X.......X.................X.......................] duration:
5.68333505s
FAIL (2025-08-21 20:01:07 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.683364888s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
6.05185004s
FAIL (2025-08-21 20:00:39 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 6.052092002s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..............X..................X..........................] duration:
1.865544949s
FAIL (2025-08-21 20:00:31 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.865575335s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...............X..................X.........................] duration:
2.14054249s
FAIL (2025-08-21 20:00:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.140571376s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.X.X.....X.X.......X.................X......................] duration:
6.038702643s
FAIL (2025-08-21 20:00:05 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.038733634s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
5.802542256s
FAIL (2025-08-21 19:59:50 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.802570621s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [..X.X.....X.X.......X.................X.....................] duration:
6.143351643s
FAIL (2025-08-21 19:59:47 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.143397161s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [................X..................X........................] duration:
2.012163712s
FAIL (2025-08-21 19:59:32 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.012194951s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
5.731814388s
FAIL (2025-08-21 19:58:55 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.731835277s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [...X.X.....X.X.......X.................X....................] duration:
6.109063632s
FAIL (2025-08-21 19:58:52 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.109124207s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................X..................X.......................] duration:
1.948478842s
FAIL (2025-08-21 19:58:36 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.948508408s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [....X.X.....X.X.......X.................X...................] duration:
5.99266877s
FAIL (2025-08-21 19:58:01 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.99270001s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
5.967559166s
FAIL (2025-08-21 19:57:55 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.967591789s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..................X..................X......................] duration:
1.906524163s
FAIL (2025-08-21 19:57:22 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.906555764s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.....X.X.....X.X.......X.................X..................] duration:
6.006595251s
FAIL (2025-08-21 19:57:09 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.006621712s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.886231757s
FAIL (2025-08-21 19:57:03 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.88625395s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [......X.X.....X.X.......X.................X.................] duration:
5.988210055s
FAIL (2025-08-21 19:56:47 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.988236366s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
5.678068353s
FAIL (2025-08-21 19:56:27 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.678092349s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...................X..................X.....................] duration:
1.88083496s
FAIL (2025-08-21 19:56:14 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.88086108s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.......X.X.....X.X.......X.................X................] duration:
5.960001761s
FAIL (2025-08-21 19:55:59 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.960025227s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
5.935598666s
FAIL (2025-08-21 19:55:46 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.935625066s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....................X..................X....................] duration:
1.954024219s
FAIL (2025-08-21 19:55:45 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.954049628s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....................X..................X...................] duration:
1.996839441s
FAIL (2025-08-21 19:55:02 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.99687029s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [........X.X.....X.X.......X.................X...............] duration:
5.77328096s
FAIL (2025-08-21 19:54:57 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.773303604s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
6.178699329s
FAIL (2025-08-21 19:54:54 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 6.178727252s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......................X..................X..................] duration:
1.859727655s
FAIL (2025-08-21 19:54:24 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.859776909s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
5.898304646s
FAIL (2025-08-21 19:54:13 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.898327931s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
7 out of 60 is failed (threshold=3) [X........X.X.....X.X.......X.................X..............] duration:
6.087989006s
FAIL (2025-08-21 19:53:50 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.088019246s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
5.704039647s
FAIL (2025-08-21 19:53:18 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.704060166s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......................X..................X.................] duration:
1.980719068s
FAIL (2025-08-21 19:53:15 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.980747343s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
6.201461933s
FAIL (2025-08-21 19:52:57 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 6.201488985s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
7 out of 60 is failed (threshold=3) [.X........X.X.....X.X.......X.................X.............] duration:
6.093912696s
FAIL (2025-08-21 19:52:47 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.093936101s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
5.903551156s
FAIL (2025-08-21 19:52:22 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.903573579s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
7 out of 60 is failed (threshold=3) [..X........X.X.....X.X.......X.................X............] duration:
5.968429347s
FAIL (2025-08-21 19:52:13 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.968459495s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [........................X..................X................] duration:
1.931657935s
FAIL (2025-08-21 19:52:09 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.931685218s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
7 out of 60 is failed (threshold=3) [...X........X.X.....X.X.......X.................X...........] duration:
5.778392319s
FAIL (2025-08-21 19:51:46 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:43:22.884337203 (314h17m17.315417324s) [upload=326975]: Starting upload Aug 21 19:43:22.884393010 (314h17m17.315473131s) [upload=326975]: Done starting upload Aug 21 19:43:22.884456912 (314h17m17.315537043s) [upload=326975]: Uploading segments... Aug 21 19:43:22.884623320 (314h17m17.315703451s) [upload=326975]: Got next segment. Inline: false Aug 21 19:43:22.884653899 (314h17m17.31573403s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:43:22.884658407 (314h17m17.315738538s) [upload=326975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 19:43:23.102820154 (314h17m17.533900295s) [upload=326975]: Next segment err: metaclient: Too Many Requests Aug 21 19:43:23.102842577 (314h17m17.533922708s) [upload=326975]: Done uploading segments... Aug 21 19:43:23.102856203 (314h17m17.533936334s) [upload=326975]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.778414812s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.........................X..................X...............] duration:
2.005770259s
FAIL (2025-08-21 19:51:42 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 19:40:12.249532717 (314h14m6.680612848s) [upload=326925]: Starting upload Aug 21 19:40:12.249577243 (314h14m6.680657374s) [upload=326925]: Done starting upload Aug 21 19:40:12.249649992 (314h14m6.680730123s) [upload=326925]: Uploading segments... Aug 21 19:40:12.249692132 (314h14m6.680772263s) [upload=326925]: Got next segment. Inline: true Aug 21 19:40:12.249696551 (314h14m6.680776682s) [upload=326925]: Waiting for error group managing segments... Aug 21 19:40:12.249729434 (314h14m6.680809565s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 19:40:12.249733512 (314h14m6.680813633s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 19:40:12.249736247 (314h14m6.680816368s) [upload=326925]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 19:40:12.467506024 (314h14m6.898586145s) [upload=326925]: Done uploading segments... Aug 21 19:40:12.467535571 (314h14m6.898615702s) [upload=326925]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.005795729s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
5.665151897s
FAIL (2025-08-21 19:51:26 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.665178408s
core
uplink
us1
|