Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
5.927301333s
FAIL (2025-08-28 05:04:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.927326571s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [...............X....XX.............X............X...........] duration:
1.882162187s
FAIL (2025-08-28 05:04:42 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 28 04:56:58.117958301 (11h18m37.280559239s) [upload=11853]: Starting upload
Aug 28 04:56:58.118004119 (11h18m37.280605057s) [upload=11853]: Done starting upload
Aug 28 04:56:58.118069954 (11h18m37.280670892s) [upload=11853]: Uploading segments...
Aug 28 04:56:58.118159085 (11h18m37.280760013s) [upload=11853]: Got next segment. Inline: true
Aug 28 04:56:58.118164655 (11h18m37.280765583s) [upload=11853]: Waiting for error group managing segments...
Aug 28 04:56:58.118221174 (11h18m37.280822112s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 28 04:56:58.118226324 (11h18m37.280827262s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 28 04:56:58.118229139 (11h18m37.280830077s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 28 04:56:58.419987015 (11h18m37.582587953s) [upload=11853]: Done uploading segments...
Aug 28 04:56:58.420004739 (11h18m37.582605677s) [upload=11853]: Upload finished. err: <nil>
Duration: 1.88219495s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [................X....XX.............X............X..........] duration:
1.99042489s
FAIL (2025-08-28 05:04:03 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 28 04:56:58.117958301 (11h18m37.280559239s) [upload=11853]: Starting upload
Aug 28 04:56:58.118004119 (11h18m37.280605057s) [upload=11853]: Done starting upload
Aug 28 04:56:58.118069954 (11h18m37.280670892s) [upload=11853]: Uploading segments...
Aug 28 04:56:58.118159085 (11h18m37.280760013s) [upload=11853]: Got next segment. Inline: true
Aug 28 04:56:58.118164655 (11h18m37.280765583s) [upload=11853]: Waiting for error group managing segments...
Aug 28 04:56:58.118221174 (11h18m37.280822112s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 28 04:56:58.118226324 (11h18m37.280827262s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 28 04:56:58.118229139 (11h18m37.280830077s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 28 04:56:58.419987015 (11h18m37.582587953s) [upload=11853]: Done uploading segments...
Aug 28 04:56:58.420004739 (11h18m37.582605677s) [upload=11853]: Upload finished. err: <nil>
Duration: 1.9904562s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
6.058637889s
FAIL (2025-08-28 05:03:57 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.058669079s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [.................X....XX.............X............X.........] duration:
2.147908775s
FAIL (2025-08-28 05:03:48 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 28 04:56:58.117958301 (11h18m37.280559239s) [upload=11853]: Starting upload
Aug 28 04:56:58.118004119 (11h18m37.280605057s) [upload=11853]: Done starting upload
Aug 28 04:56:58.118069954 (11h18m37.280670892s) [upload=11853]: Uploading segments...
Aug 28 04:56:58.118159085 (11h18m37.280760013s) [upload=11853]: Got next segment. Inline: true
Aug 28 04:56:58.118164655 (11h18m37.280765583s) [upload=11853]: Waiting for error group managing segments...
Aug 28 04:56:58.118221174 (11h18m37.280822112s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 28 04:56:58.118226324 (11h18m37.280827262s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 28 04:56:58.118229139 (11h18m37.280830077s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 28 04:56:58.419987015 (11h18m37.582587953s) [upload=11853]: Done uploading segments...
Aug 28 04:56:58.420004739 (11h18m37.582605677s) [upload=11853]: Upload finished. err: <nil>
Duration: 2.1479378s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
5.903600602s
FAIL (2025-08-28 05:03:22 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.903631139s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
5.894219265s
FAIL (2025-08-28 05:02:52 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.894282337s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [..................X....XX.............X............X........] duration:
2.460857539s
FAIL (2025-08-28 05:02:41 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 28 04:56:58.117958301 (11h18m37.280559239s) [upload=11853]: Starting upload
Aug 28 04:56:58.118004119 (11h18m37.280605057s) [upload=11853]: Done starting upload
Aug 28 04:56:58.118069954 (11h18m37.280670892s) [upload=11853]: Uploading segments...
Aug 28 04:56:58.118159085 (11h18m37.280760013s) [upload=11853]: Got next segment. Inline: true
Aug 28 04:56:58.118164655 (11h18m37.280765583s) [upload=11853]: Waiting for error group managing segments...
Aug 28 04:56:58.118221174 (11h18m37.280822112s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 28 04:56:58.118226324 (11h18m37.280827262s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 28 04:56:58.118229139 (11h18m37.280830077s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 28 04:56:58.419987015 (11h18m37.582587953s) [upload=11853]: Done uploading segments...
Aug 28 04:56:58.420004739 (11h18m37.582605677s) [upload=11853]: Upload finished. err: <nil>
Duration: 2.460887766s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
6.302695668s
FAIL (2025-08-28 05:02:20 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.302724923s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [...................X....XX.............X............X.......] duration:
2.059930262s
FAIL (2025-08-28 05:02:17 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 28 04:56:58.117958301 (11h18m37.280559239s) [upload=11853]: Starting upload
Aug 28 04:56:58.118004119 (11h18m37.280605057s) [upload=11853]: Done starting upload
Aug 28 04:56:58.118069954 (11h18m37.280670892s) [upload=11853]: Uploading segments...
Aug 28 04:56:58.118159085 (11h18m37.280760013s) [upload=11853]: Got next segment. Inline: true
Aug 28 04:56:58.118164655 (11h18m37.280765583s) [upload=11853]: Waiting for error group managing segments...
Aug 28 04:56:58.118221174 (11h18m37.280822112s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 28 04:56:58.118226324 (11h18m37.280827262s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 28 04:56:58.118229139 (11h18m37.280830077s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 28 04:56:58.419987015 (11h18m37.582587953s) [upload=11853]: Done uploading segments...
Aug 28 04:56:58.420004739 (11h18m37.582605677s) [upload=11853]: Upload finished. err: <nil>
Duration: 2.059963125s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [....................X....XX.............X............X......] duration:
2.006078705s
FAIL (2025-08-28 05:01:38 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 28 04:56:58.117958301 (11h18m37.280559239s) [upload=11853]: Starting upload
Aug 28 04:56:58.118004119 (11h18m37.280605057s) [upload=11853]: Done starting upload
Aug 28 04:56:58.118069954 (11h18m37.280670892s) [upload=11853]: Uploading segments...
Aug 28 04:56:58.118159085 (11h18m37.280760013s) [upload=11853]: Got next segment. Inline: true
Aug 28 04:56:58.118164655 (11h18m37.280765583s) [upload=11853]: Waiting for error group managing segments...
Aug 28 04:56:58.118221174 (11h18m37.280822112s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 28 04:56:58.118226324 (11h18m37.280827262s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 28 04:56:58.118229139 (11h18m37.280830077s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 28 04:56:58.419987015 (11h18m37.582587953s) [upload=11853]: Done uploading segments...
Aug 28 04:56:58.420004739 (11h18m37.582605677s) [upload=11853]: Upload finished. err: <nil>
Duration: 2.006103854s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
5.975456707s
FAIL (2025-08-28 05:01:33 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.975482517s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [.....................X....XX.............X............X.....] duration:
2.069012596s
FAIL (2025-08-28 05:01:12 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 28 04:56:58.117958301 (11h18m37.280559239s) [upload=11853]: Starting upload
Aug 28 04:56:58.118004119 (11h18m37.280605057s) [upload=11853]: Done starting upload
Aug 28 04:56:58.118069954 (11h18m37.280670892s) [upload=11853]: Uploading segments...
Aug 28 04:56:58.118159085 (11h18m37.280760013s) [upload=11853]: Got next segment. Inline: true
Aug 28 04:56:58.118164655 (11h18m37.280765583s) [upload=11853]: Waiting for error group managing segments...
Aug 28 04:56:58.118221174 (11h18m37.280822112s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 28 04:56:58.118226324 (11h18m37.280827262s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 28 04:56:58.118229139 (11h18m37.280830077s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 28 04:56:58.419987015 (11h18m37.582587953s) [upload=11853]: Done uploading segments...
Aug 28 04:56:58.420004739 (11h18m37.582605677s) [upload=11853]: Upload finished. err: <nil>
Duration: 2.069035349s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
6.231468572s
FAIL (2025-08-28 05:01:11 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.231494852s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [......................X....XX.............X............X....] duration:
2.166489273s
FAIL (2025-08-28 05:00:39 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 28 04:56:58.117958301 (11h18m37.280559239s) [upload=11853]: Starting upload
Aug 28 04:56:58.118004119 (11h18m37.280605057s) [upload=11853]: Done starting upload
Aug 28 04:56:58.118069954 (11h18m37.280670892s) [upload=11853]: Uploading segments...
Aug 28 04:56:58.118159085 (11h18m37.280760013s) [upload=11853]: Got next segment. Inline: true
Aug 28 04:56:58.118164655 (11h18m37.280765583s) [upload=11853]: Waiting for error group managing segments...
Aug 28 04:56:58.118221174 (11h18m37.280822112s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 28 04:56:58.118226324 (11h18m37.280827262s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 28 04:56:58.118229139 (11h18m37.280830077s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 28 04:56:58.419987015 (11h18m37.582587953s) [upload=11853]: Done uploading segments...
Aug 28 04:56:58.420004739 (11h18m37.582605677s) [upload=11853]: Upload finished. err: <nil>
Duration: 2.16651899s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
6.313036184s
FAIL (2025-08-28 05:00:30 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.313064368s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
5.921413878s
FAIL (2025-08-28 05:00:04 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.921445209s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [.......................X....XX.............X............X...] duration:
2.146917259s
FAIL (2025-08-28 04:59:42 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 28 04:56:58.117958301 (11h18m37.280559239s) [upload=11853]: Starting upload
Aug 28 04:56:58.118004119 (11h18m37.280605057s) [upload=11853]: Done starting upload
Aug 28 04:56:58.118069954 (11h18m37.280670892s) [upload=11853]: Uploading segments...
Aug 28 04:56:58.118159085 (11h18m37.280760013s) [upload=11853]: Got next segment. Inline: true
Aug 28 04:56:58.118164655 (11h18m37.280765583s) [upload=11853]: Waiting for error group managing segments...
Aug 28 04:56:58.118221174 (11h18m37.280822112s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 28 04:56:58.118226324 (11h18m37.280827262s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 28 04:56:58.118229139 (11h18m37.280830077s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 28 04:56:58.419987015 (11h18m37.582587953s) [upload=11853]: Done uploading segments...
Aug 28 04:56:58.420004739 (11h18m37.582605677s) [upload=11853]: Upload finished. err: <nil>
Duration: 2.146952889s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
6.213683713s
FAIL (2025-08-28 04:59:07 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.213717459s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [........................X....XX.............X............X..] duration:
2.033458986s
FAIL (2025-08-28 04:58:41 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 28 04:56:58.117958301 (11h18m37.280559239s) [upload=11853]: Starting upload
Aug 28 04:56:58.118004119 (11h18m37.280605057s) [upload=11853]: Done starting upload
Aug 28 04:56:58.118069954 (11h18m37.280670892s) [upload=11853]: Uploading segments...
Aug 28 04:56:58.118159085 (11h18m37.280760013s) [upload=11853]: Got next segment. Inline: true
Aug 28 04:56:58.118164655 (11h18m37.280765583s) [upload=11853]: Waiting for error group managing segments...
Aug 28 04:56:58.118221174 (11h18m37.280822112s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 28 04:56:58.118226324 (11h18m37.280827262s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 28 04:56:58.118229139 (11h18m37.280830077s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 28 04:56:58.419987015 (11h18m37.582587953s) [upload=11853]: Done uploading segments...
Aug 28 04:56:58.420004739 (11h18m37.582605677s) [upload=11853]: Upload finished. err: <nil>
Duration: 2.033497239s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
5.948129044s
FAIL (2025-08-28 04:57:52 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.948181656s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [.........................X....XX.............X............X.] duration:
2.060073209s
FAIL (2025-08-28 04:57:50 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 28 04:56:58.117958301 (11h18m37.280559239s) [upload=11853]: Starting upload
Aug 28 04:56:58.118004119 (11h18m37.280605057s) [upload=11853]: Done starting upload
Aug 28 04:56:58.118069954 (11h18m37.280670892s) [upload=11853]: Uploading segments...
Aug 28 04:56:58.118159085 (11h18m37.280760013s) [upload=11853]: Got next segment. Inline: true
Aug 28 04:56:58.118164655 (11h18m37.280765583s) [upload=11853]: Waiting for error group managing segments...
Aug 28 04:56:58.118221174 (11h18m37.280822112s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 28 04:56:58.118226324 (11h18m37.280827262s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 28 04:56:58.118229139 (11h18m37.280830077s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 28 04:56:58.419987015 (11h18m37.582587953s) [upload=11853]: Done uploading segments...
Aug 28 04:56:58.420004739 (11h18m37.582605677s) [upload=11853]: Upload finished. err: <nil>
Duration: 2.060096023s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [..........................X....XX.............X............X] duration:
1.755015168s
FAIL (2025-08-28 04:56:59 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 28 04:56:58.117958301 (11h18m37.280559239s) [upload=11853]: Starting upload
Aug 28 04:56:58.118004119 (11h18m37.280605057s) [upload=11853]: Done starting upload
Aug 28 04:56:58.118069954 (11h18m37.280670892s) [upload=11853]: Uploading segments...
Aug 28 04:56:58.118159085 (11h18m37.280760013s) [upload=11853]: Got next segment. Inline: true
Aug 28 04:56:58.118164655 (11h18m37.280765583s) [upload=11853]: Waiting for error group managing segments...
Aug 28 04:56:58.118221174 (11h18m37.280822112s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 28 04:56:58.118226324 (11h18m37.280827262s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 28 04:56:58.118229139 (11h18m37.280830077s) [upload=11853]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 28 04:56:58.419987015 (11h18m37.582587953s) [upload=11853]: Done uploading segments...
Aug 28 04:56:58.420004739 (11h18m37.582605677s) [upload=11853]: Upload finished. err: <nil>
Duration: 1.755050125s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
5.844169782s
FAIL (2025-08-28 04:56:45 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.844199118s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.971540591s
FAIL (2025-08-28 04:56:23 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.971570148s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
4.899526545s
FAIL (2025-08-28 04:56:19 UTC)
orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded
Duration: 4.899567864s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [...........................X....XX.............X............] duration:
2.190632195s
FAIL (2025-08-28 04:55:50 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 28 04:46:59.449245569 (11h8m38.611846507s) [upload=11675]: Starting upload Aug 28 04:46:59.449289362 (11h8m38.6118903s) [upload=11675]: Done starting upload Aug 28 04:46:59.449340019 (11h8m38.611940957s) [upload=11675]: Uploading segments... Aug 28 04:46:59.449379194 (11h8m38.611980172s) [upload=11675]: Got next segment. Inline: true Aug 28 04:46:59.449385006 (11h8m38.611985964s) [upload=11675]: Waiting for error group managing segments... Aug 28 04:46:59.449435422 (11h8m38.61203636s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:46:59.449439871 (11h8m38.612040809s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 28 04:46:59.449442255 (11h8m38.612043193s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 28 04:46:59.921602935 (11h8m39.084203873s) [upload=11675]: Done uploading segments... Aug 28 04:46:59.921620379 (11h8m39.084221317s) [upload=11675]: Upload finished. err: <nil>
Duration: 2.190660298s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
5.683103909s
FAIL (2025-08-28 04:55:37 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.683129227s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
4.800540827s
FAIL (2025-08-28 04:55:27 UTC)
orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded
Duration: 4.800559663s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
6.012738587s
FAIL (2025-08-28 04:55:22 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.012768003s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [............................X....XX.............X...........] duration:
2.208330082s
FAIL (2025-08-28 04:54:51 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 28 04:46:59.449245569 (11h8m38.611846507s) [upload=11675]: Starting upload Aug 28 04:46:59.449289362 (11h8m38.6118903s) [upload=11675]: Done starting upload Aug 28 04:46:59.449340019 (11h8m38.611940957s) [upload=11675]: Uploading segments... Aug 28 04:46:59.449379194 (11h8m38.611980172s) [upload=11675]: Got next segment. Inline: true Aug 28 04:46:59.449385006 (11h8m38.611985964s) [upload=11675]: Waiting for error group managing segments... Aug 28 04:46:59.449435422 (11h8m38.61203636s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:46:59.449439871 (11h8m38.612040809s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 28 04:46:59.449442255 (11h8m38.612043193s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 28 04:46:59.921602935 (11h8m39.084203873s) [upload=11675]: Done uploading segments... Aug 28 04:46:59.921620379 (11h8m39.084221317s) [upload=11675]: Upload finished. err: <nil>
Duration: 2.208355609s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.............................X....XX.............X..........] duration:
2.14524093s
FAIL (2025-08-28 04:54:30 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 28 04:46:59.449245569 (11h8m38.611846507s) [upload=11675]: Starting upload Aug 28 04:46:59.449289362 (11h8m38.6118903s) [upload=11675]: Done starting upload Aug 28 04:46:59.449340019 (11h8m38.611940957s) [upload=11675]: Uploading segments... Aug 28 04:46:59.449379194 (11h8m38.611980172s) [upload=11675]: Got next segment. Inline: true Aug 28 04:46:59.449385006 (11h8m38.611985964s) [upload=11675]: Waiting for error group managing segments... Aug 28 04:46:59.449435422 (11h8m38.61203636s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:46:59.449439871 (11h8m38.612040809s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 28 04:46:59.449442255 (11h8m38.612043193s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 28 04:46:59.921602935 (11h8m39.084203873s) [upload=11675]: Done uploading segments... Aug 28 04:46:59.921620379 (11h8m39.084221317s) [upload=11675]: Upload finished. err: <nil>
Duration: 2.145275246s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
6.048498278s
FAIL (2025-08-28 04:54:27 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.048524679s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
5.172172533s
FAIL (2025-08-28 04:54:16 UTC)
orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded
Duration: 5.172200808s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..............................X....XX.............X.........] duration:
1.988337153s
FAIL (2025-08-28 04:54:09 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 28 04:46:59.449245569 (11h8m38.611846507s) [upload=11675]: Starting upload Aug 28 04:46:59.449289362 (11h8m38.6118903s) [upload=11675]: Done starting upload Aug 28 04:46:59.449340019 (11h8m38.611940957s) [upload=11675]: Uploading segments... Aug 28 04:46:59.449379194 (11h8m38.611980172s) [upload=11675]: Got next segment. Inline: true Aug 28 04:46:59.449385006 (11h8m38.611985964s) [upload=11675]: Waiting for error group managing segments... Aug 28 04:46:59.449435422 (11h8m38.61203636s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:46:59.449439871 (11h8m38.612040809s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 28 04:46:59.449442255 (11h8m38.612043193s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 28 04:46:59.921602935 (11h8m39.084203873s) [upload=11675]: Done uploading segments... Aug 28 04:46:59.921620379 (11h8m39.084221317s) [upload=11675]: Upload finished. err: <nil>
Duration: 1.988362991s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [X.....................................................X.....] duration:
6.447891056s
FAIL (2025-08-28 04:53:52 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.447913649s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
6.005019728s
FAIL (2025-08-28 04:53:47 UTC)
orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded
Duration: 6.005041519s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [X..............................X....XX.............X........] duration:
2.032245964s
FAIL (2025-08-28 04:53:24 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 28 04:46:59.449245569 (11h8m38.611846507s) [upload=11675]: Starting upload Aug 28 04:46:59.449289362 (11h8m38.6118903s) [upload=11675]: Done starting upload Aug 28 04:46:59.449340019 (11h8m38.611940957s) [upload=11675]: Uploading segments... Aug 28 04:46:59.449379194 (11h8m38.611980172s) [upload=11675]: Got next segment. Inline: true Aug 28 04:46:59.449385006 (11h8m38.611985964s) [upload=11675]: Waiting for error group managing segments... Aug 28 04:46:59.449435422 (11h8m38.61203636s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:46:59.449439871 (11h8m38.612040809s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 28 04:46:59.449442255 (11h8m38.612043193s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 28 04:46:59.921602935 (11h8m39.084203873s) [upload=11675]: Done uploading segments... Aug 28 04:46:59.921620379 (11h8m39.084221317s) [upload=11675]: Upload finished. err: <nil>
Duration: 2.032284266s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.X.....................................................X....] duration:
6.014321579s
FAIL (2025-08-28 04:53:17 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.014344983s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
4.909537981s
FAIL (2025-08-28 04:52:41 UTC)
orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded
Duration: 4.909560162s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [.X..............................X....XX.............X.......] duration:
2.511230078s
FAIL (2025-08-28 04:52:36 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 28 04:46:59.449245569 (11h8m38.611846507s) [upload=11675]: Starting upload Aug 28 04:46:59.449289362 (11h8m38.6118903s) [upload=11675]: Done starting upload Aug 28 04:46:59.449340019 (11h8m38.611940957s) [upload=11675]: Uploading segments... Aug 28 04:46:59.449379194 (11h8m38.611980172s) [upload=11675]: Got next segment. Inline: true Aug 28 04:46:59.449385006 (11h8m38.611985964s) [upload=11675]: Waiting for error group managing segments... Aug 28 04:46:59.449435422 (11h8m38.61203636s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:46:59.449439871 (11h8m38.612040809s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 28 04:46:59.449442255 (11h8m38.612043193s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 28 04:46:59.921602935 (11h8m39.084203873s) [upload=11675]: Done uploading segments... Aug 28 04:46:59.921620379 (11h8m39.084221317s) [upload=11675]: Upload finished. err: <nil>
Duration: 2.511259855s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X.....................................................X...] duration:
6.019930277s
FAIL (2025-08-28 04:52:17 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.019951777s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [X....X......................................................] duration:
6.87099299s
FAIL (2025-08-28 04:52:10 UTC)
orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded
Duration: 6.871018099s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.X....X.....................................................] duration:
5.18105998s
FAIL (2025-08-28 04:51:48 UTC)
orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded
Duration: 5.181085489s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...X.....................................................X..] duration:
5.982940481s
FAIL (2025-08-28 04:51:47 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.982965859s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [..X..............................X....XX.............X......] duration:
2.010614933s
FAIL (2025-08-28 04:51:33 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 28 04:46:59.449245569 (11h8m38.611846507s) [upload=11675]: Starting upload Aug 28 04:46:59.449289362 (11h8m38.6118903s) [upload=11675]: Done starting upload Aug 28 04:46:59.449340019 (11h8m38.611940957s) [upload=11675]: Uploading segments... Aug 28 04:46:59.449379194 (11h8m38.611980172s) [upload=11675]: Got next segment. Inline: true Aug 28 04:46:59.449385006 (11h8m38.611985964s) [upload=11675]: Waiting for error group managing segments... Aug 28 04:46:59.449435422 (11h8m38.61203636s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:46:59.449439871 (11h8m38.612040809s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 28 04:46:59.449442255 (11h8m38.612043193s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 28 04:46:59.921602935 (11h8m39.084203873s) [upload=11675]: Done uploading segments... Aug 28 04:46:59.921620379 (11h8m39.084221317s) [upload=11675]: Upload finished. err: <nil>
Duration: 2.010641605s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [....X.....................................................X.] duration:
6.266340027s
FAIL (2025-08-28 04:50:46 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 28 04:50:15.570177084 (11h11m54.732778022s) [upload=11736]: Starting upload Aug 28 04:50:15.570241136 (11h11m54.732842084s) [upload=11736]: Done starting upload Aug 28 04:50:15.570337070 (11h11m54.732938018s) [upload=11736]: Uploading segments... Aug 28 04:50:15.570598440 (11h11m54.733199378s) [upload=11736]: Got next segment. Inline: false Aug 28 04:50:15.570637885 (11h11m54.733238823s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:50:15.570642695 (11h11m54.733243633s) [upload=11736]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 28 04:50:15.813083150 (11h11m54.975684098s) [upload=11736]: Next segment err: metaclient: Too Many Requests Aug 28 04:50:15.813105944 (11h11m54.975706882s) [upload=11736]: Done uploading segments... Aug 28 04:50:15.813122274 (11h11m54.975723212s) [upload=11736]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.266370575s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
5 out of 60 is failed (threshold=3) [...X..............................X....XX.............X.....] duration:
2.21390856s
FAIL (2025-08-28 04:50:45 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 28 04:46:59.449245569 (11h8m38.611846507s) [upload=11675]: Starting upload Aug 28 04:46:59.449289362 (11h8m38.6118903s) [upload=11675]: Done starting upload Aug 28 04:46:59.449340019 (11h8m38.611940957s) [upload=11675]: Uploading segments... Aug 28 04:46:59.449379194 (11h8m38.611980172s) [upload=11675]: Got next segment. Inline: true Aug 28 04:46:59.449385006 (11h8m38.611985964s) [upload=11675]: Waiting for error group managing segments... Aug 28 04:46:59.449435422 (11h8m38.61203636s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:46:59.449439871 (11h8m38.612040809s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 28 04:46:59.449442255 (11h8m38.612043193s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 28 04:46:59.921602935 (11h8m39.084203873s) [upload=11675]: Done uploading segments... Aug 28 04:46:59.921620379 (11h8m39.084221317s) [upload=11675]: Upload finished. err: <nil>
Duration: 2.213928058s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
3 out of 60 is failed (threshold=3) [X.X....X....................................................] duration:
4.87105394s
FAIL (2025-08-28 04:50:45 UTC)
orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded
Duration: 4.871096853s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
6 out of 60 is failed (threshold=3) [X...X..............................X....XX.............X....] duration:
2.468815139s
FAIL (2025-08-28 04:50:31 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 28 04:46:59.449245569 (11h8m38.611846507s) [upload=11675]: Starting upload Aug 28 04:46:59.449289362 (11h8m38.6118903s) [upload=11675]: Done starting upload Aug 28 04:46:59.449340019 (11h8m38.611940957s) [upload=11675]: Uploading segments... Aug 28 04:46:59.449379194 (11h8m38.611980172s) [upload=11675]: Got next segment. Inline: true Aug 28 04:46:59.449385006 (11h8m38.611985964s) [upload=11675]: Waiting for error group managing segments... Aug 28 04:46:59.449435422 (11h8m38.61203636s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 28 04:46:59.449439871 (11h8m38.612040809s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 28 04:46:59.449442255 (11h8m38.612043193s) [upload=11675]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 28 04:46:59.921602935 (11h8m39.084203873s) [upload=11675]: Done uploading segments... Aug 28 04:46:59.921620379 (11h8m39.084221317s) [upload=11675]: Upload finished. err: <nil>
Duration: 2.468847572s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|