Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
5.98140736s
FAIL (2025-08-16 18:06:27 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.981429302s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
5.723950986s
FAIL (2025-08-16 18:06:22 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.723984772s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [X...........................X............X..................] duration:
1.948027789s
FAIL (2025-08-16 18:06:08 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:49:35.329029567 (192h23m29.760109698s) [upload=200137]: Starting upload Aug 16 17:49:35.329076697 (192h23m29.760156818s) [upload=200137]: Done starting upload Aug 16 17:49:35.329127524 (192h23m29.760207656s) [upload=200137]: Uploading segments... Aug 16 17:49:35.329164635 (192h23m29.760244766s) [upload=200137]: Got next segment. Inline: true Aug 16 17:49:35.329168442 (192h23m29.760248573s) [upload=200137]: Waiting for error group managing segments... Aug 16 17:49:35.329200844 (192h23m29.760280975s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:49:35.329204882 (192h23m29.760285013s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 17:49:35.329207356 (192h23m29.760287487s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 17:49:35.626479733 (192h23m30.057559864s) [upload=200137]: Done uploading segments... Aug 16 17:49:35.626508198 (192h23m30.057588329s) [upload=200137]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.948057055s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [X......................X....................................] duration:
6.006478096s
FAIL (2025-08-16 18:05:13 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.006507714s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.615224985s
FAIL (2025-08-16 18:05:09 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.61526892s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.X......................X...................................] duration:
6.130882709s
FAIL (2025-08-16 18:04:58 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.130908228s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.X...........................X............X.................] duration:
1.960134652s
FAIL (2025-08-16 18:04:55 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:49:35.329029567 (192h23m29.760109698s) [upload=200137]: Starting upload Aug 16 17:49:35.329076697 (192h23m29.760156818s) [upload=200137]: Done starting upload Aug 16 17:49:35.329127524 (192h23m29.760207656s) [upload=200137]: Uploading segments... Aug 16 17:49:35.329164635 (192h23m29.760244766s) [upload=200137]: Got next segment. Inline: true Aug 16 17:49:35.329168442 (192h23m29.760248573s) [upload=200137]: Waiting for error group managing segments... Aug 16 17:49:35.329200844 (192h23m29.760280975s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:49:35.329204882 (192h23m29.760285013s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 17:49:35.329207356 (192h23m29.760287487s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 17:49:35.626479733 (192h23m30.057559864s) [upload=200137]: Done uploading segments... Aug 16 17:49:35.626508198 (192h23m30.057588329s) [upload=200137]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.960158778s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
5.821410427s
FAIL (2025-08-16 18:04:42 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.821438671s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X......................X..................................] duration:
6.157710358s
FAIL (2025-08-16 18:04:34 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.157730997s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...X......................X.................................] duration:
5.990279577s
FAIL (2025-08-16 18:03:59 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.990305055s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [X.X...........................X............X................] duration:
1.891086306s
FAIL (2025-08-16 18:03:59 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:49:35.329029567 (192h23m29.760109698s) [upload=200137]: Starting upload Aug 16 17:49:35.329076697 (192h23m29.760156818s) [upload=200137]: Done starting upload Aug 16 17:49:35.329127524 (192h23m29.760207656s) [upload=200137]: Uploading segments... Aug 16 17:49:35.329164635 (192h23m29.760244766s) [upload=200137]: Got next segment. Inline: true Aug 16 17:49:35.329168442 (192h23m29.760248573s) [upload=200137]: Waiting for error group managing segments... Aug 16 17:49:35.329200844 (192h23m29.760280975s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:49:35.329204882 (192h23m29.760285013s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 17:49:35.329207356 (192h23m29.760287487s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 17:49:35.626479733 (192h23m30.057559864s) [upload=200137]: Done uploading segments... Aug 16 17:49:35.626508198 (192h23m30.057588329s) [upload=200137]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.891117015s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
5.630470982s
FAIL (2025-08-16 18:03:56 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.630503202s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [....X......................X................................] duration:
5.854305968s
FAIL (2025-08-16 18:03:33 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.854332036s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
5.766165787s
FAIL (2025-08-16 18:03:12 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.766194563s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.X.X...........................X............X...............] duration:
2.097462319s
FAIL (2025-08-16 18:03:07 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:49:35.329029567 (192h23m29.760109698s) [upload=200137]: Starting upload Aug 16 17:49:35.329076697 (192h23m29.760156818s) [upload=200137]: Done starting upload Aug 16 17:49:35.329127524 (192h23m29.760207656s) [upload=200137]: Uploading segments... Aug 16 17:49:35.329164635 (192h23m29.760244766s) [upload=200137]: Got next segment. Inline: true Aug 16 17:49:35.329168442 (192h23m29.760248573s) [upload=200137]: Waiting for error group managing segments... Aug 16 17:49:35.329200844 (192h23m29.760280975s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:49:35.329204882 (192h23m29.760285013s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 17:49:35.329207356 (192h23m29.760287487s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 17:49:35.626479733 (192h23m30.057559864s) [upload=200137]: Done uploading segments... Aug 16 17:49:35.626508198 (192h23m30.057588329s) [upload=200137]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.097502225s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
6.077972803s
FAIL (2025-08-16 18:02:55 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 6.078005055s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
6.074068272s
FAIL (2025-08-16 18:02:33 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 6.07409863s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....X......................X...............................] duration:
6.187041984s
FAIL (2025-08-16 18:02:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.187061381s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..X.X...........................X............X..............] duration:
1.867947875s
FAIL (2025-08-16 18:02:15 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:49:35.329029567 (192h23m29.760109698s) [upload=200137]: Starting upload Aug 16 17:49:35.329076697 (192h23m29.760156818s) [upload=200137]: Done starting upload Aug 16 17:49:35.329127524 (192h23m29.760207656s) [upload=200137]: Uploading segments... Aug 16 17:49:35.329164635 (192h23m29.760244766s) [upload=200137]: Got next segment. Inline: true Aug 16 17:49:35.329168442 (192h23m29.760248573s) [upload=200137]: Waiting for error group managing segments... Aug 16 17:49:35.329200844 (192h23m29.760280975s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:49:35.329204882 (192h23m29.760285013s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 17:49:35.329207356 (192h23m29.760287487s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 17:49:35.626479733 (192h23m30.057559864s) [upload=200137]: Done uploading segments... Aug 16 17:49:35.626508198 (192h23m30.057588329s) [upload=200137]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.86797553s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
6.105224474s
FAIL (2025-08-16 18:01:36 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 6.105248619s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [......X......................X..............................] duration:
6.11650167s
FAIL (2025-08-16 18:01:36 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.116527089s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [...X.X...........................X............X.............] duration:
2.117991166s
FAIL (2025-08-16 18:01:10 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:49:35.329029567 (192h23m29.760109698s) [upload=200137]: Starting upload Aug 16 17:49:35.329076697 (192h23m29.760156818s) [upload=200137]: Done starting upload Aug 16 17:49:35.329127524 (192h23m29.760207656s) [upload=200137]: Uploading segments... Aug 16 17:49:35.329164635 (192h23m29.760244766s) [upload=200137]: Got next segment. Inline: true Aug 16 17:49:35.329168442 (192h23m29.760248573s) [upload=200137]: Waiting for error group managing segments... Aug 16 17:49:35.329200844 (192h23m29.760280975s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:49:35.329204882 (192h23m29.760285013s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 17:49:35.329207356 (192h23m29.760287487s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 17:49:35.626479733 (192h23m30.057559864s) [upload=200137]: Done uploading segments... Aug 16 17:49:35.626508198 (192h23m30.057588329s) [upload=200137]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.118011585s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......X......................X.............................] duration:
6.110554232s
FAIL (2025-08-16 18:01:09 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.110579009s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.742383753s
FAIL (2025-08-16 18:00:56 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.742411357s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [........X......................X............................] duration:
6.163290949s
FAIL (2025-08-16 18:00:37 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.163320055s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [....X.X...........................X............X............] duration:
2.014954102s
FAIL (2025-08-16 18:00:29 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:49:35.329029567 (192h23m29.760109698s) [upload=200137]: Starting upload Aug 16 17:49:35.329076697 (192h23m29.760156818s) [upload=200137]: Done starting upload Aug 16 17:49:35.329127524 (192h23m29.760207656s) [upload=200137]: Uploading segments... Aug 16 17:49:35.329164635 (192h23m29.760244766s) [upload=200137]: Got next segment. Inline: true Aug 16 17:49:35.329168442 (192h23m29.760248573s) [upload=200137]: Waiting for error group managing segments... Aug 16 17:49:35.329200844 (192h23m29.760280975s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:49:35.329204882 (192h23m29.760285013s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 17:49:35.329207356 (192h23m29.760287487s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 17:49:35.626479733 (192h23m30.057559864s) [upload=200137]: Done uploading segments... Aug 16 17:49:35.626508198 (192h23m30.057588329s) [upload=200137]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.014998776s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
5.996048242s
FAIL (2025-08-16 18:00:27 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.996075623s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
5.684787003s
FAIL (2025-08-16 18:00:11 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.684827973s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........X......................X...........................] duration:
6.240756723s
FAIL (2025-08-16 17:59:37 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.240778203s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
5.909065117s
FAIL (2025-08-16 17:59:33 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.909094874s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.....X.X...........................X............X...........] duration:
2.057154876s
FAIL (2025-08-16 17:59:31 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:49:35.329029567 (192h23m29.760109698s) [upload=200137]: Starting upload Aug 16 17:49:35.329076697 (192h23m29.760156818s) [upload=200137]: Done starting upload Aug 16 17:49:35.329127524 (192h23m29.760207656s) [upload=200137]: Uploading segments... Aug 16 17:49:35.329164635 (192h23m29.760244766s) [upload=200137]: Got next segment. Inline: true Aug 16 17:49:35.329168442 (192h23m29.760248573s) [upload=200137]: Waiting for error group managing segments... Aug 16 17:49:35.329200844 (192h23m29.760280975s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:49:35.329204882 (192h23m29.760285013s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 17:49:35.329207356 (192h23m29.760287487s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 17:49:35.626479733 (192h23m30.057559864s) [upload=200137]: Done uploading segments... Aug 16 17:49:35.626508198 (192h23m30.057588329s) [upload=200137]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.05718309s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
5.690612436s
FAIL (2025-08-16 17:59:15 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.690634827s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [......X.X...........................X............X..........] duration:
2.16610895s
FAIL (2025-08-16 17:59:01 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:49:35.329029567 (192h23m29.760109698s) [upload=200137]: Starting upload Aug 16 17:49:35.329076697 (192h23m29.760156818s) [upload=200137]: Done starting upload Aug 16 17:49:35.329127524 (192h23m29.760207656s) [upload=200137]: Uploading segments... Aug 16 17:49:35.329164635 (192h23m29.760244766s) [upload=200137]: Got next segment. Inline: true Aug 16 17:49:35.329168442 (192h23m29.760248573s) [upload=200137]: Waiting for error group managing segments... Aug 16 17:49:35.329200844 (192h23m29.760280975s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:49:35.329204882 (192h23m29.760285013s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 17:49:35.329207356 (192h23m29.760287487s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 17:49:35.626479733 (192h23m30.057559864s) [upload=200137]: Done uploading segments... Aug 16 17:49:35.626508198 (192h23m30.057588329s) [upload=200137]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.166137104s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........X......................X..........................] duration:
5.942977858s
FAIL (2025-08-16 17:58:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.942997205s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
5.643428757s
FAIL (2025-08-16 17:58:27 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.643457533s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.......X.X...........................X............X.........] duration:
1.877682518s
FAIL (2025-08-16 17:58:19 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:49:35.329029567 (192h23m29.760109698s) [upload=200137]: Starting upload Aug 16 17:49:35.329076697 (192h23m29.760156818s) [upload=200137]: Done starting upload Aug 16 17:49:35.329127524 (192h23m29.760207656s) [upload=200137]: Uploading segments... Aug 16 17:49:35.329164635 (192h23m29.760244766s) [upload=200137]: Got next segment. Inline: true Aug 16 17:49:35.329168442 (192h23m29.760248573s) [upload=200137]: Waiting for error group managing segments... Aug 16 17:49:35.329200844 (192h23m29.760280975s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:49:35.329204882 (192h23m29.760285013s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 17:49:35.329207356 (192h23m29.760287487s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 17:49:35.626479733 (192h23m30.057559864s) [upload=200137]: Done uploading segments... Aug 16 17:49:35.626508198 (192h23m30.057588329s) [upload=200137]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.877714549s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
5.83264096s
FAIL (2025-08-16 17:58:09 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.832672972s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........X......................X.........................] duration:
6.02110258s
FAIL (2025-08-16 17:57:41 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.021127849s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
5.596569841s
FAIL (2025-08-16 17:57:32 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.596604287s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [........X.X...........................X............X........] duration:
1.897135162s
FAIL (2025-08-16 17:57:08 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:49:35.329029567 (192h23m29.760109698s) [upload=200137]: Starting upload Aug 16 17:49:35.329076697 (192h23m29.760156818s) [upload=200137]: Done starting upload Aug 16 17:49:35.329127524 (192h23m29.760207656s) [upload=200137]: Uploading segments... Aug 16 17:49:35.329164635 (192h23m29.760244766s) [upload=200137]: Got next segment. Inline: true Aug 16 17:49:35.329168442 (192h23m29.760248573s) [upload=200137]: Waiting for error group managing segments... Aug 16 17:49:35.329200844 (192h23m29.760280975s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:49:35.329204882 (192h23m29.760285013s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 17:49:35.329207356 (192h23m29.760287487s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 17:49:35.626479733 (192h23m30.057559864s) [upload=200137]: Done uploading segments... Aug 16 17:49:35.626508198 (192h23m30.057588329s) [upload=200137]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.897154048s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [............X......................X........................] duration:
6.305588131s
FAIL (2025-08-16 17:57:03 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.305617417s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
5.459133319s
FAIL (2025-08-16 17:57:01 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.459159588s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
5.816700558s
FAIL (2025-08-16 17:56:25 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.816722751s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.........X.X...........................X............X.......] duration:
2.401439222s
FAIL (2025-08-16 17:56:17 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:49:35.329029567 (192h23m29.760109698s) [upload=200137]: Starting upload Aug 16 17:49:35.329076697 (192h23m29.760156818s) [upload=200137]: Done starting upload Aug 16 17:49:35.329127524 (192h23m29.760207656s) [upload=200137]: Uploading segments... Aug 16 17:49:35.329164635 (192h23m29.760244766s) [upload=200137]: Got next segment. Inline: true Aug 16 17:49:35.329168442 (192h23m29.760248573s) [upload=200137]: Waiting for error group managing segments... Aug 16 17:49:35.329200844 (192h23m29.760280975s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:49:35.329204882 (192h23m29.760285013s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 17:49:35.329207356 (192h23m29.760287487s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 17:49:35.626479733 (192h23m30.057559864s) [upload=200137]: Done uploading segments... Aug 16 17:49:35.626508198 (192h23m30.057588329s) [upload=200137]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.401466574s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............X......................X.......................] duration:
5.930951829s
FAIL (2025-08-16 17:55:59 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.930979241s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
5.646702119s
FAIL (2025-08-16 17:55:30 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.646728149s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..........X.X...........................X............X......] duration:
1.89034302s
FAIL (2025-08-16 17:55:10 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:49:35.329029567 (192h23m29.760109698s) [upload=200137]: Starting upload Aug 16 17:49:35.329076697 (192h23m29.760156818s) [upload=200137]: Done starting upload Aug 16 17:49:35.329127524 (192h23m29.760207656s) [upload=200137]: Uploading segments... Aug 16 17:49:35.329164635 (192h23m29.760244766s) [upload=200137]: Got next segment. Inline: true Aug 16 17:49:35.329168442 (192h23m29.760248573s) [upload=200137]: Waiting for error group managing segments... Aug 16 17:49:35.329200844 (192h23m29.760280975s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:49:35.329204882 (192h23m29.760285013s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 17:49:35.329207356 (192h23m29.760287487s) [upload=200137]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 17:49:35.626479733 (192h23m30.057559864s) [upload=200137]: Done uploading segments... Aug 16 17:49:35.626508198 (192h23m30.057588329s) [upload=200137]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.890368239s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............X......................X......................] duration:
6.263860709s
FAIL (2025-08-16 17:55:06 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.263895696s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
5.878488825s
FAIL (2025-08-16 17:54:46 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.878508984s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............X......................X.....................] duration:
6.151345585s
FAIL (2025-08-16 17:54:42 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 17:39:35.825415112 (192h13m30.256495653s) [upload=199962]: Starting upload Aug 16 17:39:35.825479976 (192h13m30.256560107s) [upload=199962]: Done starting upload Aug 16 17:39:35.825758379 (192h13m30.25683851s) [upload=199962]: Uploading segments... Aug 16 17:39:35.825855374 (192h13m30.256935505s) [upload=199962]: Got next segment. Inline: false Aug 16 17:39:35.825892436 (192h13m30.256972557s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 17:39:35.825897174 (192h13m30.256977305s) [upload=199962]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 17:39:36.039810393 (192h13m30.470890514s) [upload=199962]: Next segment err: metaclient: Too Many Requests Aug 16 17:39:36.039830731 (192h13m30.470910862s) [upload=199962]: Done uploading segments... Aug 16 17:39:36.039856651 (192h13m30.470936772s) [upload=199962]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.151372676s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|