Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
2.031373665s
FAIL (2025-08-11 05:17:10 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.031402109s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.XX..................X.....X..X.............................] duration:
8.275384507s
FAIL (2025-08-11 05:16:51 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 8.275402072s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
2.031533121s
FAIL (2025-08-11 05:16:39 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.031565582s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [..XX..................X.....X..X............................] duration:
9.508004767s
FAIL (2025-08-11 05:16:11 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 9.508046608s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
1.849023574s
FAIL (2025-08-11 05:15:30 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.849048542s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [...XX..................X.....X..X...........................] duration:
9.744804467s
FAIL (2025-08-11 05:15:00 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 9.744829345s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
1.842435859s
FAIL (2025-08-11 05:14:39 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.842507627s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
1.842676972s
FAIL (2025-08-11 05:13:55 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.842703182s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [....XX..................X.....X..X..........................] duration:
7.929066715s
FAIL (2025-08-11 05:13:49 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.929092335s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
1.86822414s
FAIL (2025-08-11 05:13:35 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.868243849s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
1.814472176s
FAIL (2025-08-11 05:13:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.814489218s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
1.832856601s
FAIL (2025-08-11 05:12:48 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.832879736s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.....XX..................X.....X..X.........................] duration:
7.625825576s
FAIL (2025-08-11 05:12:43 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.625856274s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [......XX..................X.....X..X........................] duration:
7.589330764s
FAIL (2025-08-11 05:11:51 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.589367385s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
1.835183563s
FAIL (2025-08-11 05:11:41 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.835209063s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......XX..................X.....X..X.......................] duration:
7.960920247s
FAIL (2025-08-11 05:11:19 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.960955384s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
1.861597101s
FAIL (2025-08-11 05:11:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.861616688s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [........XX..................X.....X..X......................] duration:
7.153142424s
FAIL (2025-08-11 05:10:52 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.153165689s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.........XX..................X.....X..X.....................] duration:
7.711489522s
FAIL (2025-08-11 05:10:18 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.711569365s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
1.916520919s
FAIL (2025-08-11 05:10:09 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.916548201s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
2.062858445s
FAIL (2025-08-11 05:09:44 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.062885036s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [..........XX..................X.....X..X....................] duration:
7.426961053s
FAIL (2025-08-11 05:09:16 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.427000298s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [...........XX..................X.....X..X...................] duration:
7.734744883s
FAIL (2025-08-11 05:09:00 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.734763088s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
2.010279434s
FAIL (2025-08-11 05:08:44 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.010317949s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [............XX..................X.....X..X..................] duration:
7.548559792s
FAIL (2025-08-11 05:07:49 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.548583827s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
1.826613633s
FAIL (2025-08-11 05:07:37 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.826657157s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.............XX..................X.....X..X.................] duration:
7.737782847s
FAIL (2025-08-11 05:07:18 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.737809398s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [..............XX..................X.....X..X................] duration:
7.862231222s
FAIL (2025-08-11 05:06:58 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.86225174s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
1.835990274s
FAIL (2025-08-11 05:06:27 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.836021854s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [...............XX..................X.....X..X...............] duration:
7.190657134s
FAIL (2025-08-11 05:06:03 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.190684576s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [................XX..................X.....X..X..............] duration:
8.815377333s
FAIL (2025-08-11 05:05:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 8.815451816s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.118659458s
FAIL (2025-08-11 05:05:16 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.11868585s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.099262675s
FAIL (2025-08-11 05:04:49 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.099289666s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.................XX..................X.....X..X.............] duration:
7.485404918s
FAIL (2025-08-11 05:04:41 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.485424396s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [..................XX..................X.....X..X............] duration:
8.027962487s
FAIL (2025-08-11 05:04:23 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 8.027990572s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
2.002361946s
FAIL (2025-08-11 05:03:36 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.002397444s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [...................XX..................X.....X..X...........] duration:
7.3892794s
FAIL (2025-08-11 05:03:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.389378068s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [....................XX..................X.....X..X..........] duration:
9.262988512s
FAIL (2025-08-11 05:02:26 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 9.263015974s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
1.854923309s
FAIL (2025-08-11 05:02:24 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.85495511s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.....................XX..................X.....X..X.........] duration:
6.969259833s
FAIL (2025-08-11 05:01:35 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.969291385s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
2.000203516s
FAIL (2025-08-11 05:01:19 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.000232461s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [......................XX..................X.....X..X........] duration:
8.181674716s
FAIL (2025-08-11 05:00:33 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 8.181704181s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
1.845054978s
FAIL (2025-08-11 05:00:22 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.845080918s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......................XX..................X.....X..X.......] duration:
8.094773667s
FAIL (2025-08-11 04:59:58 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 8.09480045s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
1.8148158s
FAIL (2025-08-11 04:59:46 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.814848582s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
8.249080465s
FAIL (2025-08-11 04:59:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Aug 11 04:14:05.318440227 (58h47m59.749520358s) [upload=61279]: Starting upload
Aug 11 04:14:05.318493429 (58h47m59.74957356s) [upload=61279]: Done starting upload
Aug 11 04:14:05.318549967 (58h47m59.749630098s) [upload=61279]: Uploading segments...
Aug 11 04:14:05.318671200 (58h47m59.749751331s) [upload=61279]: Got next segment. Inline: false
Aug 11 04:14:05.318697891 (58h47m59.749778022s) [upload=61279]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 04:14:05.318702129 (58h47m59.74978226s) [upload=61279]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 11 04:14:05.765239352 (58h48m0.196319493s) [seg_pos={0 0}, upload=61279]: Begin upload segment...
Aug 11 04:14:05.765296240 (58h48m0.196376371s) [seg_pos={0 0}, upload=61279]: Done begin upload segment.
Aug 11 04:14:05.765312542 (58h48m0.196392673s) [noise=true , address=45.91.116.5:4297 , piece_id=C3NVLER4JE4TTUNCC4YZ5CJXS7G7DJG5PBHWGCEOW6GRVVDX555Q, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.765316890 (58h48m0.196397021s) [noise=true , address=104.36.82.3:4767 , piece_id=Q6YEXCB3CQZYXQ54ITILSFU5LFCV5S6PVVFTMSHJ4UJG6DCMZZCQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.765670567 (58h48m0.196750688s) [noise=true , address=77.90.31.12:4592 , piece_id=VNK25A4A3N452RJCX5R67NCNMA6VXESSV3YIBBUCGBJPONXYLCRQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.765995629 (58h48m0.19707576s) [noise=true , address=45.91.116.4:4938 , piece_id=OV27IPAB35KRP5CAASYQEHFCHGQ2Z6DSKAGFRIKV237BWKCQBGBQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.766263110 (58h48m0.197343241s) [noise=true , address=104.36.82.13:4186 , piece_id=TDPQ6AKV5DR5JVDHEXCI72IGYEYVDXB3PYR7WOXUTAOXMLQYMDDA, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.766547765 (58h48m0.197627896s) [noise=true , address=45.91.116.7:4010 , piece_id=LBWCSEOSBBSJGHZWKH4D3TCBFQ67NPH5MWIVUC6CT3JNMAFLBEVQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.766877987 (58h48m0.197958118s) [noise=true , address=77.90.31.14:4663 , piece_id=XI2Q7T2ARKZIB3X5JAKGXDMVUKZ565MMEHFRTNZWLWPZC76FQPLQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.767100553 (58h48m0.198180684s) [noise=true , address=45.91.116.10:4081 , piece_id=JB2IBDZ7FMTMLUZ23JSCJWHXSUDS2ZPCMZHSY5WFCQROR7OGACOQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.767302078 (58h48m0.198382209s) [noise=true , address=77.90.31.19:4842 , piece_id=VVUBRZE4A3JKRYXW5DWYTWK6XXFYTGLW2AXCIKJWJDF4QZVUXGZA, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.767522400 (58h48m0.198602521s) [noise=true , address=104.36.82.7:4910 , piece_id=SDOK67NTIXR6HIZZVPBMAEVD4XO2ZGK7S5VJFSLRAP6JE5VZQIKA, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.767725739 (58h48m0.19880587s) [noise=true , address=45.91.116.13:4154 , piece_id=ZQL7ZRSQKV7U2VNKBCFPH5AXWQIMG445AXTAPZEPRTB7GPQKG7DA, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.767932685 (58h48m0.199012816s) [noise=true , address=77.90.31.11:4554 , piece_id=QTGF7Q4CFDYENXAFHSPW4QFCCS6YC2FF3AUFGY4KBXNJYS7JF7TQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.768173816 (58h48m0.199253957s) [noise=true , address=45.91.116.11:4119 , piece_id=GY6REQIUQ7KUKM7622B2N53FYX3OGR3WWUO5LH35MA6XDOPWBCYQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.768438583 (58h48m0.199518714s) [noise=true , address=104.36.82.10:4079 , piece_id=POPE63FDLNL7H6W6S4LAUOSEGCUV4ITOET6SV6A4YJBOLX7B5GZQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.768667711 (58h48m0.199747842s) [noise=true , address=104.36.82.8:4970 , piece_id=LWGBQF6BHVYU6V5BSYOXPQ6SDY22OFV5GTEDW6OCL5CCQPUB2WUA, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.768927238 (58h48m0.200007369s) [noise=true , address=77.90.31.16:4733 , piece_id=5PU6HVEZGTAWHFF2HK7E2ECCONEQGG6BJL5VTP7AM4C7DF2GGGCQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:08.309487348 (58h48m2.740567479s) [upload=61279]: Next returned nil segment
Aug 11 04:14:08.309537213 (58h48m2.740617344s) [upload=61279]: Waiting for error group managing segments...
Aug 11 04:14:09.205648620 (58h48m3.636728751s) [noise=true , address=104.36.82.10:4079 , piece_id=POPE63FDLNL7H6W6S4LAUOSEGCUV4ITOET6SV6A4YJBOLX7B5GZQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:09.280236001 (58h48m3.711316122s) [noise=true , address=77.90.31.14:4663 , piece_id=XI2Q7T2ARKZIB3X5JAKGXDMVUKZ565MMEHFRTNZWLWPZC76FQPLQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:09.426409640 (58h48m3.857489761s) [noise=true , address=104.36.82.13:4186 , piece_id=TDPQ6AKV5DR5JVDHEXCI72IGYEYVDXB3PYR7WOXUTAOXMLQYMDDA, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:09.536862883 (58h48m3.967943004s) [noise=true , address=45.91.116.11:4119 , piece_id=GY6REQIUQ7KUKM7622B2N53FYX3OGR3WWUO5LH35MA6XDOPWBCYQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:11.592695751 (58h48m6.023775883s) [noise=true , address=104.36.82.3:4767 , piece_id=Q6YEXCB3CQZYXQ54ITILSFU5LFCV5S6PVVFTMSHJ4UJG6DCMZZCQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:13.145726321 (58h48m7.576806452s) [noise=true , address=45.91.116.10:4081 , piece_id=JB2IBDZ7FMTMLUZ23JSCJWHXSUDS2ZPCMZHSY5WFCQROR7OGACOQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:14.206618488 (58h48m8.637698619s) [noise=true , address=45.91.116.13:4154 , piece_id=ZQL7ZRSQKV7U2VNKBCFPH5AXWQIMG445AXTAPZEPRTB7GPQKG7DA, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:18.326267188 (58h48m12.757347309s) [noise=true , address=77.90.31.11:4554 , piece_id=QTGF7Q4CFDYENXAFHSPW4QFCCS6YC2FF3AUFGY4KBXNJYS7JF7TQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:18.346485323 (58h48m12.777565454s) [noise=true , address=45.91.116.4:4938 , piece_id=OV27IPAB35KRP5CAASYQEHFCHGQ2Z6DSKAGFRIKV237BWKCQBGBQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:19.027231925 (58h48m13.458312056s) [noise=true , address=77.90.31.12:4592 , piece_id=VNK25A4A3N452RJCX5R67NCNMA6VXESSV3YIBBUCGBJPONXYLCRQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:21.696292999 (58h48m16.12737313s) [noise=true , address=45.91.116.7:4010 , piece_id=LBWCSEOSBBSJGHZWKH4D3TCBFQ67NPH5MWIVUC6CT3JNMAFLBEVQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:27.199043519 (58h48m21.63012364s) [noise=true , address=45.91.116.5:4297 , piece_id=C3NVLER4JE4TTUNCC4YZ5CJXS7G7DJG5PBHWGCEOW6GRVVDX555Q, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:34.675838024 (58h48m29.106918155s) [noise=true , address=77.90.31.16:4733 , piece_id=5PU6HVEZGTAWHFF2HK7E2ECCONEQGG6BJL5VTP7AM4C7DF2GGGCQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: expected piece hash; context deadline exceeded; EOF
Aug 11 04:14:34.675910663 (58h48m29.106990794s) [noise=true , address=104.36.82.7:4910 , piece_id=SDOK67NTIXR6HIZZVPBMAEVD4XO2ZGK7S5VJFSLRAP6JE5VZQIKA, seg_pos={0 0}, upload=61279]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: expected piece hash; context deadline exceeded; EOF
Aug 11 04:14:34.675978763 (58h48m29.107058894s) [noise=true , address=104.36.82.8:4970 , piece_id=LWGBQF6BHVYU6V5BSYOXPQ6SDY22OFV5GTEDW6OCL5CCQPUB2WUA, seg_pos={0 0}, upload=61279]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Aug 11 04:14:34.676097411 (58h48m29.107177542s) [noise=true , address=77.90.31.19:4842 , piece_id=VVUBRZE4A3JKRYXW5DWYTWK6XXFYTGLW2AXCIKJWJDF4QZVUXGZA, seg_pos={0 0}, upload=61279]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47230->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47230->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47230->77.90.31.19:4842: use of closed network connection
Aug 11 04:14:34.676133039 (58h48m29.10721317s) [seg_pos={0 0}, upload=61279]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Aug 11 04:14:34.904930895 (58h48m29.336011016s) [upload=61279]: Done uploading segments...
Aug 11 04:14:34.904958387 (58h48m29.336038518s) [upload=61279]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 8.249135579s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [........................XX..................X.....X..X......] duration:
7.748625369s
FAIL (2025-08-11 04:59:16 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.748654265s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
1.969091679s
FAIL (2025-08-11 04:58:39 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:44:24.932296741 (59h18m19.363376862s) [upload=61797]: Starting upload Aug 11 04:44:24.932375572 (59h18m19.363455703s) [upload=61797]: Done starting upload Aug 11 04:44:24.932485803 (59h18m19.363565994s) [upload=61797]: Uploading segments... Aug 11 04:44:24.932550086 (59h18m19.363630247s) [upload=61797]: Got next segment. Inline: true Aug 11 04:44:24.932557180 (59h18m19.363637331s) [upload=61797]: Waiting for error group managing segments... Aug 11 04:44:24.932610262 (59h18m19.363690453s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:44:24.932616814 (59h18m19.363696995s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 11 04:44:24.932621844 (59h18m19.363702005s) [upload=61797]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 11 04:44:25.294615282 (59h18m19.725695413s) [upload=61797]: Done uploading segments... Aug 11 04:44:25.294637325 (59h18m19.725717456s) [upload=61797]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.969131176s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
9.317300651s
FAIL (2025-08-11 04:58:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Aug 11 04:14:05.318440227 (58h47m59.749520358s) [upload=61279]: Starting upload
Aug 11 04:14:05.318493429 (58h47m59.74957356s) [upload=61279]: Done starting upload
Aug 11 04:14:05.318549967 (58h47m59.749630098s) [upload=61279]: Uploading segments...
Aug 11 04:14:05.318671200 (58h47m59.749751331s) [upload=61279]: Got next segment. Inline: false
Aug 11 04:14:05.318697891 (58h47m59.749778022s) [upload=61279]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 11 04:14:05.318702129 (58h47m59.74978226s) [upload=61279]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 11 04:14:05.765239352 (58h48m0.196319493s) [seg_pos={0 0}, upload=61279]: Begin upload segment...
Aug 11 04:14:05.765296240 (58h48m0.196376371s) [seg_pos={0 0}, upload=61279]: Done begin upload segment.
Aug 11 04:14:05.765312542 (58h48m0.196392673s) [noise=true , address=45.91.116.5:4297 , piece_id=C3NVLER4JE4TTUNCC4YZ5CJXS7G7DJG5PBHWGCEOW6GRVVDX555Q, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.765316890 (58h48m0.196397021s) [noise=true , address=104.36.82.3:4767 , piece_id=Q6YEXCB3CQZYXQ54ITILSFU5LFCV5S6PVVFTMSHJ4UJG6DCMZZCQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.765670567 (58h48m0.196750688s) [noise=true , address=77.90.31.12:4592 , piece_id=VNK25A4A3N452RJCX5R67NCNMA6VXESSV3YIBBUCGBJPONXYLCRQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.765995629 (58h48m0.19707576s) [noise=true , address=45.91.116.4:4938 , piece_id=OV27IPAB35KRP5CAASYQEHFCHGQ2Z6DSKAGFRIKV237BWKCQBGBQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.766263110 (58h48m0.197343241s) [noise=true , address=104.36.82.13:4186 , piece_id=TDPQ6AKV5DR5JVDHEXCI72IGYEYVDXB3PYR7WOXUTAOXMLQYMDDA, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.766547765 (58h48m0.197627896s) [noise=true , address=45.91.116.7:4010 , piece_id=LBWCSEOSBBSJGHZWKH4D3TCBFQ67NPH5MWIVUC6CT3JNMAFLBEVQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.766877987 (58h48m0.197958118s) [noise=true , address=77.90.31.14:4663 , piece_id=XI2Q7T2ARKZIB3X5JAKGXDMVUKZ565MMEHFRTNZWLWPZC76FQPLQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.767100553 (58h48m0.198180684s) [noise=true , address=45.91.116.10:4081 , piece_id=JB2IBDZ7FMTMLUZ23JSCJWHXSUDS2ZPCMZHSY5WFCQROR7OGACOQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.767302078 (58h48m0.198382209s) [noise=true , address=77.90.31.19:4842 , piece_id=VVUBRZE4A3JKRYXW5DWYTWK6XXFYTGLW2AXCIKJWJDF4QZVUXGZA, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.767522400 (58h48m0.198602521s) [noise=true , address=104.36.82.7:4910 , piece_id=SDOK67NTIXR6HIZZVPBMAEVD4XO2ZGK7S5VJFSLRAP6JE5VZQIKA, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.767725739 (58h48m0.19880587s) [noise=true , address=45.91.116.13:4154 , piece_id=ZQL7ZRSQKV7U2VNKBCFPH5AXWQIMG445AXTAPZEPRTB7GPQKG7DA, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.767932685 (58h48m0.199012816s) [noise=true , address=77.90.31.11:4554 , piece_id=QTGF7Q4CFDYENXAFHSPW4QFCCS6YC2FF3AUFGY4KBXNJYS7JF7TQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.768173816 (58h48m0.199253957s) [noise=true , address=45.91.116.11:4119 , piece_id=GY6REQIUQ7KUKM7622B2N53FYX3OGR3WWUO5LH35MA6XDOPWBCYQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.768438583 (58h48m0.199518714s) [noise=true , address=104.36.82.10:4079 , piece_id=POPE63FDLNL7H6W6S4LAUOSEGCUV4ITOET6SV6A4YJBOLX7B5GZQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.768667711 (58h48m0.199747842s) [noise=true , address=104.36.82.8:4970 , piece_id=LWGBQF6BHVYU6V5BSYOXPQ6SDY22OFV5GTEDW6OCL5CCQPUB2WUA, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:05.768927238 (58h48m0.200007369s) [noise=true , address=77.90.31.16:4733 , piece_id=5PU6HVEZGTAWHFF2HK7E2ECCONEQGG6BJL5VTP7AM4C7DF2GGGCQ, seg_pos={0 0}, upload=61279]: Uploading piece...
Aug 11 04:14:08.309487348 (58h48m2.740567479s) [upload=61279]: Next returned nil segment
Aug 11 04:14:08.309537213 (58h48m2.740617344s) [upload=61279]: Waiting for error group managing segments...
Aug 11 04:14:09.205648620 (58h48m3.636728751s) [noise=true , address=104.36.82.10:4079 , piece_id=POPE63FDLNL7H6W6S4LAUOSEGCUV4ITOET6SV6A4YJBOLX7B5GZQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:09.280236001 (58h48m3.711316122s) [noise=true , address=77.90.31.14:4663 , piece_id=XI2Q7T2ARKZIB3X5JAKGXDMVUKZ565MMEHFRTNZWLWPZC76FQPLQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:09.426409640 (58h48m3.857489761s) [noise=true , address=104.36.82.13:4186 , piece_id=TDPQ6AKV5DR5JVDHEXCI72IGYEYVDXB3PYR7WOXUTAOXMLQYMDDA, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:09.536862883 (58h48m3.967943004s) [noise=true , address=45.91.116.11:4119 , piece_id=GY6REQIUQ7KUKM7622B2N53FYX3OGR3WWUO5LH35MA6XDOPWBCYQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:11.592695751 (58h48m6.023775883s) [noise=true , address=104.36.82.3:4767 , piece_id=Q6YEXCB3CQZYXQ54ITILSFU5LFCV5S6PVVFTMSHJ4UJG6DCMZZCQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:13.145726321 (58h48m7.576806452s) [noise=true , address=45.91.116.10:4081 , piece_id=JB2IBDZ7FMTMLUZ23JSCJWHXSUDS2ZPCMZHSY5WFCQROR7OGACOQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:14.206618488 (58h48m8.637698619s) [noise=true , address=45.91.116.13:4154 , piece_id=ZQL7ZRSQKV7U2VNKBCFPH5AXWQIMG445AXTAPZEPRTB7GPQKG7DA, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:18.326267188 (58h48m12.757347309s) [noise=true , address=77.90.31.11:4554 , piece_id=QTGF7Q4CFDYENXAFHSPW4QFCCS6YC2FF3AUFGY4KBXNJYS7JF7TQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:18.346485323 (58h48m12.777565454s) [noise=true , address=45.91.116.4:4938 , piece_id=OV27IPAB35KRP5CAASYQEHFCHGQ2Z6DSKAGFRIKV237BWKCQBGBQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:19.027231925 (58h48m13.458312056s) [noise=true , address=77.90.31.12:4592 , piece_id=VNK25A4A3N452RJCX5R67NCNMA6VXESSV3YIBBUCGBJPONXYLCRQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:21.696292999 (58h48m16.12737313s) [noise=true , address=45.91.116.7:4010 , piece_id=LBWCSEOSBBSJGHZWKH4D3TCBFQ67NPH5MWIVUC6CT3JNMAFLBEVQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:27.199043519 (58h48m21.63012364s) [noise=true , address=45.91.116.5:4297 , piece_id=C3NVLER4JE4TTUNCC4YZ5CJXS7G7DJG5PBHWGCEOW6GRVVDX555Q, seg_pos={0 0}, upload=61279]: Done uploading piece. err: <nil>
Aug 11 04:14:34.675838024 (58h48m29.106918155s) [noise=true , address=77.90.31.16:4733 , piece_id=5PU6HVEZGTAWHFF2HK7E2ECCONEQGG6BJL5VTP7AM4C7DF2GGGCQ, seg_pos={0 0}, upload=61279]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: expected piece hash; context deadline exceeded; EOF
Aug 11 04:14:34.675910663 (58h48m29.106990794s) [noise=true , address=104.36.82.7:4910 , piece_id=SDOK67NTIXR6HIZZVPBMAEVD4XO2ZGK7S5VJFSLRAP6JE5VZQIKA, seg_pos={0 0}, upload=61279]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: expected piece hash; context deadline exceeded; EOF
Aug 11 04:14:34.675978763 (58h48m29.107058894s) [noise=true , address=104.36.82.8:4970 , piece_id=LWGBQF6BHVYU6V5BSYOXPQ6SDY22OFV5GTEDW6OCL5CCQPUB2WUA, seg_pos={0 0}, upload=61279]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: expected piece hash; context deadline exceeded; EOF
Aug 11 04:14:34.676097411 (58h48m29.107177542s) [noise=true , address=77.90.31.19:4842 , piece_id=VVUBRZE4A3JKRYXW5DWYTWK6XXFYTGLW2AXCIKJWJDF4QZVUXGZA, seg_pos={0 0}, upload=61279]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47230->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47230->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47230->77.90.31.19:4842: use of closed network connection
Aug 11 04:14:34.676133039 (58h48m29.10721317s) [seg_pos={0 0}, upload=61279]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Aug 11 04:14:34.904930895 (58h48m29.336011016s) [upload=61279]: Done uploading segments...
Aug 11 04:14:34.904958387 (58h48m29.336038518s) [upload=61279]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 9.317323945s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.........................XX..................X.....X..X.....] duration:
8.18194308s
FAIL (2025-08-11 04:58:06 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 11 04:53:15.975483199 (59h27m10.40656332s) [upload=61957]: Starting upload Aug 11 04:53:15.975529377 (59h27m10.406609498s) [upload=61957]: Done starting upload Aug 11 04:53:15.975620010 (59h27m10.406700141s) [upload=61957]: Uploading segments... Aug 11 04:53:15.975802861 (59h27m10.406883002s) [upload=61957]: Got next segment. Inline: false Aug 11 04:53:15.975845712 (59h27m10.406925843s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 11 04:53:15.975853518 (59h27m10.406933689s) [upload=61957]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 11 04:53:16.197092986 (59h27m10.628173157s) [upload=61957]: Next segment err: metaclient: Too Many Requests Aug 11 04:53:16.197121811 (59h27m10.628201952s) [upload=61957]: Done uploading segments... Aug 11 04:53:16.197141379 (59h27m10.62822157s) [upload=61957]: Upload finished. err: metaclient: Too Many Requests
Duration: 8.182022744s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|