Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [......................XX.X..............X...........X.......] duration:
6.097107311s
FAIL (2025-08-21 23:16:49 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.097150813s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......................XX.X..............X...........X......] duration:
6.313735565s
FAIL (2025-08-21 23:16:30 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.313764942s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................................XX.......................] duration:
5.845710222s
FAIL (2025-08-21 23:16:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.845726464s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [............................................X...X........XX.] duration:
1.890691858s
FAIL (2025-08-21 23:16:17 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 21 23:15:31.385933556 (317h49m25.817013687s) [upload=330657]: Starting upload
Aug 21 23:15:31.385974795 (317h49m25.817054926s) [upload=330657]: Done starting upload
Aug 21 23:15:31.386029720 (317h49m25.817109851s) [upload=330657]: Uploading segments...
Aug 21 23:15:31.386080717 (317h49m25.817160858s) [upload=330657]: Got next segment. Inline: true
Aug 21 23:15:31.386086278 (317h49m25.817166399s) [upload=330657]: Waiting for error group managing segments...
Aug 21 23:15:31.386140312 (317h49m25.817220473s) [upload=330657]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 23:15:31.386147325 (317h49m25.817227466s) [upload=330657]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 23:15:31.386151864 (317h49m25.817231995s) [upload=330657]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 23:15:32.310207456 (317h49m26.741287587s) [upload=330657]: Done uploading segments...
Aug 21 23:15:32.310225530 (317h49m26.741305651s) [upload=330657]: Upload finished. err: <nil>
Duration: 1.890733958s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................................XX......................] duration:
5.828951736s
FAIL (2025-08-21 23:15:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.828973177s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.............................................X...X........XX] duration:
2.262061659s
FAIL (2025-08-21 23:15:32 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 21 23:15:31.385933556 (317h49m25.817013687s) [upload=330657]: Starting upload
Aug 21 23:15:31.385974795 (317h49m25.817054926s) [upload=330657]: Done starting upload
Aug 21 23:15:31.386029720 (317h49m25.817109851s) [upload=330657]: Uploading segments...
Aug 21 23:15:31.386080717 (317h49m25.817160858s) [upload=330657]: Got next segment. Inline: true
Aug 21 23:15:31.386086278 (317h49m25.817166399s) [upload=330657]: Waiting for error group managing segments...
Aug 21 23:15:31.386140312 (317h49m25.817220473s) [upload=330657]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 23:15:31.386147325 (317h49m25.817227466s) [upload=330657]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 23:15:31.386151864 (317h49m25.817231995s) [upload=330657]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 23:15:32.310207456 (317h49m26.741287587s) [upload=330657]: Done uploading segments...
Aug 21 23:15:32.310225530 (317h49m26.741305651s) [upload=330657]: Upload finished. err: <nil>
Duration: 2.262085324s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [X.......................XX.X..............X...........X.....] duration:
6.315339857s
FAIL (2025-08-21 23:15:19 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.315371359s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..............................................X...X........X] duration:
1.846524239s
FAIL (2025-08-21 23:15:02 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 4b0d10fb40c651e29b98cb1f4df79f7f2e000256f28c879b6a867ea52185b4df != before ec0f6d2b1707aa78b0ddce4ad77dc660aaa2539bcb6ee78e840e8ffeeb57a324 ======LOGS====== Aug 21 23:15:01.526959122 (317h48m55.958039263s) [upload=330648]: Starting upload Aug 21 23:15:01.527018896 (317h48m55.958099027s) [upload=330648]: Done starting upload Aug 21 23:15:01.527078861 (317h48m55.958158992s) [upload=330648]: Uploading segments... Aug 21 23:15:01.527179814 (317h48m55.958259945s) [upload=330648]: Got next segment. Inline: true Aug 21 23:15:01.527185214 (317h48m55.958265346s) [upload=330648]: Waiting for error group managing segments... Aug 21 23:15:01.527229640 (317h48m55.958309781s) [upload=330648]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:15:01.527235000 (317h48m55.958315141s) [upload=330648]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:15:01.527238246 (317h48m55.958318387s) [upload=330648]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:15:01.949743380 (317h48m56.380823521s) [upload=330648]: Done uploading segments... Aug 21 23:15:01.949769680 (317h48m56.380849812s) [upload=330648]: Upload finished. err: <nil>
Duration: 1.846554577s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration:
6.306408725s
FAIL (2025-08-21 23:14:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.30643258s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...............................................X...X........] duration:
1.864165914s
FAIL (2025-08-21 23:14:13 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f40bab4c6b348263f32829a2e9ff4c40b8983560b6b1c433d69ca20dd112c026 != before f41d84d19a1edcb87f112df3535839214bb8f76ff915229ba96d842ba0facf2e ======LOGS====== Aug 21 23:09:11.595051937 (317h43m6.026132068s) [upload=330544]: Starting upload Aug 21 23:09:11.595100651 (317h43m6.026180792s) [upload=330544]: Done starting upload Aug 21 23:09:11.595150146 (317h43m6.026230267s) [upload=330544]: Uploading segments... Aug 21 23:09:11.595182868 (317h43m6.026262999s) [upload=330544]: Got next segment. Inline: true Aug 21 23:09:11.595186715 (317h43m6.026266846s) [upload=330544]: Waiting for error group managing segments... Aug 21 23:09:11.595219377 (317h43m6.026299509s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:09:11.595223155 (317h43m6.026303286s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:09:11.595225499 (317h43m6.02630563s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:09:11.901358048 (317h43m6.332438169s) [upload=330544]: Done uploading segments... Aug 21 23:09:11.901374480 (317h43m6.332454611s) [upload=330544]: Upload finished. err: <nil>
Duration: 1.864192945s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.X.......................XX.X..............X...........X....] duration:
6.019790092s
FAIL (2025-08-21 23:14:05 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.019823287s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................................XX....................] duration:
5.781541972s
FAIL (2025-08-21 23:13:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.781563053s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [................................................X...X.......] duration:
1.861226665s
FAIL (2025-08-21 23:13:42 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f40bab4c6b348263f32829a2e9ff4c40b8983560b6b1c433d69ca20dd112c026 != before f41d84d19a1edcb87f112df3535839214bb8f76ff915229ba96d842ba0facf2e ======LOGS====== Aug 21 23:09:11.595051937 (317h43m6.026132068s) [upload=330544]: Starting upload Aug 21 23:09:11.595100651 (317h43m6.026180792s) [upload=330544]: Done starting upload Aug 21 23:09:11.595150146 (317h43m6.026230267s) [upload=330544]: Uploading segments... Aug 21 23:09:11.595182868 (317h43m6.026262999s) [upload=330544]: Got next segment. Inline: true Aug 21 23:09:11.595186715 (317h43m6.026266846s) [upload=330544]: Waiting for error group managing segments... Aug 21 23:09:11.595219377 (317h43m6.026299509s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:09:11.595223155 (317h43m6.026303286s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:09:11.595225499 (317h43m6.02630563s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:09:11.901358048 (317h43m6.332438169s) [upload=330544]: Done uploading segments... Aug 21 23:09:11.901374480 (317h43m6.332454611s) [upload=330544]: Upload finished. err: <nil>
Duration: 1.861255469s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [..X.......................XX.X..............X...........X...] duration:
5.885037254s
FAIL (2025-08-21 23:13:21 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.885056681s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [...X.......................XX.X..............X...........X..] duration:
6.15886567s
FAIL (2025-08-21 23:12:55 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.158898151s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................................XX...................] duration:
6.225159984s
FAIL (2025-08-21 23:12:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.225183859s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................................................X...X......] duration:
2.104500087s
FAIL (2025-08-21 23:12:43 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f40bab4c6b348263f32829a2e9ff4c40b8983560b6b1c433d69ca20dd112c026 != before f41d84d19a1edcb87f112df3535839214bb8f76ff915229ba96d842ba0facf2e ======LOGS====== Aug 21 23:09:11.595051937 (317h43m6.026132068s) [upload=330544]: Starting upload Aug 21 23:09:11.595100651 (317h43m6.026180792s) [upload=330544]: Done starting upload Aug 21 23:09:11.595150146 (317h43m6.026230267s) [upload=330544]: Uploading segments... Aug 21 23:09:11.595182868 (317h43m6.026262999s) [upload=330544]: Got next segment. Inline: true Aug 21 23:09:11.595186715 (317h43m6.026266846s) [upload=330544]: Waiting for error group managing segments... Aug 21 23:09:11.595219377 (317h43m6.026299509s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:09:11.595223155 (317h43m6.026303286s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:09:11.595225499 (317h43m6.02630563s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:09:11.901358048 (317h43m6.332438169s) [upload=330544]: Done uploading segments... Aug 21 23:09:11.901374480 (317h43m6.332454611s) [upload=330544]: Upload finished. err: <nil>
Duration: 2.104532678s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [....X.......................XX.X..............X...........X.] duration:
6.088317431s
FAIL (2025-08-21 23:12:37 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.088342539s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................................XX..................] duration:
6.145606747s
FAIL (2025-08-21 23:12:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.145642436s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..................................................X...X.....] duration:
1.862463948s
FAIL (2025-08-21 23:12:14 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f40bab4c6b348263f32829a2e9ff4c40b8983560b6b1c433d69ca20dd112c026 != before f41d84d19a1edcb87f112df3535839214bb8f76ff915229ba96d842ba0facf2e ======LOGS====== Aug 21 23:09:11.595051937 (317h43m6.026132068s) [upload=330544]: Starting upload Aug 21 23:09:11.595100651 (317h43m6.026180792s) [upload=330544]: Done starting upload Aug 21 23:09:11.595150146 (317h43m6.026230267s) [upload=330544]: Uploading segments... Aug 21 23:09:11.595182868 (317h43m6.026262999s) [upload=330544]: Got next segment. Inline: true Aug 21 23:09:11.595186715 (317h43m6.026266846s) [upload=330544]: Waiting for error group managing segments... Aug 21 23:09:11.595219377 (317h43m6.026299509s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:09:11.595223155 (317h43m6.026303286s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:09:11.595225499 (317h43m6.02630563s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:09:11.901358048 (317h43m6.332438169s) [upload=330544]: Done uploading segments... Aug 21 23:09:11.901374480 (317h43m6.332454611s) [upload=330544]: Upload finished. err: <nil>
Duration: 1.862523562s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [X..................................................X...X....] duration:
2.276538872s
FAIL (2025-08-21 23:11:58 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f40bab4c6b348263f32829a2e9ff4c40b8983560b6b1c433d69ca20dd112c026 != before f41d84d19a1edcb87f112df3535839214bb8f76ff915229ba96d842ba0facf2e ======LOGS====== Aug 21 23:09:11.595051937 (317h43m6.026132068s) [upload=330544]: Starting upload Aug 21 23:09:11.595100651 (317h43m6.026180792s) [upload=330544]: Done starting upload Aug 21 23:09:11.595150146 (317h43m6.026230267s) [upload=330544]: Uploading segments... Aug 21 23:09:11.595182868 (317h43m6.026262999s) [upload=330544]: Got next segment. Inline: true Aug 21 23:09:11.595186715 (317h43m6.026266846s) [upload=330544]: Waiting for error group managing segments... Aug 21 23:09:11.595219377 (317h43m6.026299509s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:09:11.595223155 (317h43m6.026303286s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:09:11.595225499 (317h43m6.02630563s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:09:11.901358048 (317h43m6.332438169s) [upload=330544]: Done uploading segments... Aug 21 23:09:11.901374480 (317h43m6.332454611s) [upload=330544]: Upload finished. err: <nil>
Duration: 2.276569309s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.....X.......................XX.X..............X...........X] duration:
869.235119ms
FAIL (2025-08-21 23:11:30 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 869.255159ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.X..................................................X...X...] duration:
2.105636067s
FAIL (2025-08-21 23:11:12 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f40bab4c6b348263f32829a2e9ff4c40b8983560b6b1c433d69ca20dd112c026 != before f41d84d19a1edcb87f112df3535839214bb8f76ff915229ba96d842ba0facf2e ======LOGS====== Aug 21 23:09:11.595051937 (317h43m6.026132068s) [upload=330544]: Starting upload Aug 21 23:09:11.595100651 (317h43m6.026180792s) [upload=330544]: Done starting upload Aug 21 23:09:11.595150146 (317h43m6.026230267s) [upload=330544]: Uploading segments... Aug 21 23:09:11.595182868 (317h43m6.026262999s) [upload=330544]: Got next segment. Inline: true Aug 21 23:09:11.595186715 (317h43m6.026266846s) [upload=330544]: Waiting for error group managing segments... Aug 21 23:09:11.595219377 (317h43m6.026299509s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:09:11.595223155 (317h43m6.026303286s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:09:11.595225499 (317h43m6.02630563s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:09:11.901358048 (317h43m6.332438169s) [upload=330544]: Done uploading segments... Aug 21 23:09:11.901374480 (317h43m6.332454611s) [upload=330544]: Upload finished. err: <nil>
Duration: 2.105661405s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................................XX.................] duration:
6.216542765s
FAIL (2025-08-21 23:11:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.216559997s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [X.....X.......................XX.X..............X...........] duration:
5.826024878s
FAIL (2025-08-21 23:10:52 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:01:51.826406781 (317h35m46.257486902s) [upload=330420]: Starting upload Aug 21 23:01:51.826449382 (317h35m46.257529503s) [upload=330420]: Done starting upload Aug 21 23:01:51.826636801 (317h35m46.257716932s) [upload=330420]: Uploading segments... Aug 21 23:01:51.826754176 (317h35m46.257834317s) [upload=330420]: Got next segment. Inline: false Aug 21 23:01:51.826826554 (317h35m46.257906695s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:01:51.826832295 (317h35m46.257912426s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:01:52.039215545 (317h35m46.470295676s) [upload=330420]: Next segment err: metaclient: Too Many Requests Aug 21 23:01:52.039237608 (317h35m46.470317729s) [upload=330420]: Done uploading segments... Aug 21 23:01:52.039249991 (317h35m46.470330122s) [upload=330420]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.826048955s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..X..................................................X...X..] duration:
1.890817939s
FAIL (2025-08-21 23:10:37 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f40bab4c6b348263f32829a2e9ff4c40b8983560b6b1c433d69ca20dd112c026 != before f41d84d19a1edcb87f112df3535839214bb8f76ff915229ba96d842ba0facf2e ======LOGS====== Aug 21 23:09:11.595051937 (317h43m6.026132068s) [upload=330544]: Starting upload Aug 21 23:09:11.595100651 (317h43m6.026180792s) [upload=330544]: Done starting upload Aug 21 23:09:11.595150146 (317h43m6.026230267s) [upload=330544]: Uploading segments... Aug 21 23:09:11.595182868 (317h43m6.026262999s) [upload=330544]: Got next segment. Inline: true Aug 21 23:09:11.595186715 (317h43m6.026266846s) [upload=330544]: Waiting for error group managing segments... Aug 21 23:09:11.595219377 (317h43m6.026299509s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:09:11.595223155 (317h43m6.026303286s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:09:11.595225499 (317h43m6.02630563s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:09:11.901358048 (317h43m6.332438169s) [upload=330544]: Done uploading segments... Aug 21 23:09:11.901374480 (317h43m6.332454611s) [upload=330544]: Upload finished. err: <nil>
Duration: 1.890838568s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...X..................................................X...X.] duration:
1.997741522s
FAIL (2025-08-21 23:10:01 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f40bab4c6b348263f32829a2e9ff4c40b8983560b6b1c433d69ca20dd112c026 != before f41d84d19a1edcb87f112df3535839214bb8f76ff915229ba96d842ba0facf2e ======LOGS====== Aug 21 23:09:11.595051937 (317h43m6.026132068s) [upload=330544]: Starting upload Aug 21 23:09:11.595100651 (317h43m6.026180792s) [upload=330544]: Done starting upload Aug 21 23:09:11.595150146 (317h43m6.026230267s) [upload=330544]: Uploading segments... Aug 21 23:09:11.595182868 (317h43m6.026262999s) [upload=330544]: Got next segment. Inline: true Aug 21 23:09:11.595186715 (317h43m6.026266846s) [upload=330544]: Waiting for error group managing segments... Aug 21 23:09:11.595219377 (317h43m6.026299509s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:09:11.595223155 (317h43m6.026303286s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:09:11.595225499 (317h43m6.02630563s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:09:11.901358048 (317h43m6.332438169s) [upload=330544]: Done uploading segments... Aug 21 23:09:11.901374480 (317h43m6.332454611s) [upload=330544]: Upload finished. err: <nil>
Duration: 1.997765548s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................................XX................] duration:
5.927623268s
FAIL (2025-08-21 23:09:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.927647244s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.X.....X.......................XX.X..............X..........] duration:
6.088858812s
FAIL (2025-08-21 23:09:56 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:01:51.826406781 (317h35m46.257486902s) [upload=330420]: Starting upload Aug 21 23:01:51.826449382 (317h35m46.257529503s) [upload=330420]: Done starting upload Aug 21 23:01:51.826636801 (317h35m46.257716932s) [upload=330420]: Uploading segments... Aug 21 23:01:51.826754176 (317h35m46.257834317s) [upload=330420]: Got next segment. Inline: false Aug 21 23:01:51.826826554 (317h35m46.257906695s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:01:51.826832295 (317h35m46.257912426s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:01:52.039215545 (317h35m46.470295676s) [upload=330420]: Next segment err: metaclient: Too Many Requests Aug 21 23:01:52.039237608 (317h35m46.470317729s) [upload=330420]: Done uploading segments... Aug 21 23:01:52.039249991 (317h35m46.470330122s) [upload=330420]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.088884931s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [..X.....X.......................XX.X..............X.........] duration:
6.183492507s
FAIL (2025-08-21 23:09:23 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:01:51.826406781 (317h35m46.257486902s) [upload=330420]: Starting upload Aug 21 23:01:51.826449382 (317h35m46.257529503s) [upload=330420]: Done starting upload Aug 21 23:01:51.826636801 (317h35m46.257716932s) [upload=330420]: Uploading segments... Aug 21 23:01:51.826754176 (317h35m46.257834317s) [upload=330420]: Got next segment. Inline: false Aug 21 23:01:51.826826554 (317h35m46.257906695s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:01:51.826832295 (317h35m46.257912426s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:01:52.039215545 (317h35m46.470295676s) [upload=330420]: Next segment err: metaclient: Too Many Requests Aug 21 23:01:52.039237608 (317h35m46.470317729s) [upload=330420]: Done uploading segments... Aug 21 23:01:52.039249991 (317h35m46.470330122s) [upload=330420]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.183517515s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [....X..................................................X...X] duration:
1.677871852s
FAIL (2025-08-21 23:09:12 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after f40bab4c6b348263f32829a2e9ff4c40b8983560b6b1c433d69ca20dd112c026 != before f41d84d19a1edcb87f112df3535839214bb8f76ff915229ba96d842ba0facf2e ======LOGS====== Aug 21 23:09:11.595051937 (317h43m6.026132068s) [upload=330544]: Starting upload Aug 21 23:09:11.595100651 (317h43m6.026180792s) [upload=330544]: Done starting upload Aug 21 23:09:11.595150146 (317h43m6.026230267s) [upload=330544]: Uploading segments... Aug 21 23:09:11.595182868 (317h43m6.026262999s) [upload=330544]: Got next segment. Inline: true Aug 21 23:09:11.595186715 (317h43m6.026266846s) [upload=330544]: Waiting for error group managing segments... Aug 21 23:09:11.595219377 (317h43m6.026299509s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:09:11.595223155 (317h43m6.026303286s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:09:11.595225499 (317h43m6.02630563s) [upload=330544]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:09:11.901358048 (317h43m6.332438169s) [upload=330544]: Done uploading segments... Aug 21 23:09:11.901374480 (317h43m6.332454611s) [upload=330544]: Upload finished. err: <nil>
Duration: 1.677896038s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........................................XX...............] duration:
5.900880438s
FAIL (2025-08-21 23:09:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.900902781s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [...X.....X.......................XX.X..............X........] duration:
6.161379137s
FAIL (2025-08-21 23:08:31 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:01:51.826406781 (317h35m46.257486902s) [upload=330420]: Starting upload Aug 21 23:01:51.826449382 (317h35m46.257529503s) [upload=330420]: Done starting upload Aug 21 23:01:51.826636801 (317h35m46.257716932s) [upload=330420]: Uploading segments... Aug 21 23:01:51.826754176 (317h35m46.257834317s) [upload=330420]: Got next segment. Inline: false Aug 21 23:01:51.826826554 (317h35m46.257906695s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:01:51.826832295 (317h35m46.257912426s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:01:52.039215545 (317h35m46.470295676s) [upload=330420]: Next segment err: metaclient: Too Many Requests Aug 21 23:01:52.039237608 (317h35m46.470317729s) [upload=330420]: Done uploading segments... Aug 21 23:01:52.039249991 (317h35m46.470330122s) [upload=330420]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.161404154s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....X..................................................X...] duration:
2.35052906s
FAIL (2025-08-21 23:08:24 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 21 23:05:24.685747594 (317h39m19.116827755s) [upload=330479]: Starting upload
Aug 21 23:05:24.685873967 (317h39m19.116954098s) [upload=330479]: Done starting upload
Aug 21 23:05:24.685941395 (317h39m19.117021526s) [upload=330479]: Uploading segments...
Aug 21 23:05:24.686009706 (317h39m19.117089837s) [upload=330479]: Got next segment. Inline: true
Aug 21 23:05:24.686014836 (317h39m19.117094967s) [upload=330479]: Waiting for error group managing segments...
Aug 21 23:05:24.686047238 (317h39m19.117127359s) [upload=330479]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 23:05:24.686051566 (317h39m19.117131697s) [upload=330479]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 23:05:24.686054061 (317h39m19.117134192s) [upload=330479]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 23:05:24.990932300 (317h39m19.422012431s) [upload=330479]: Done uploading segments...
Aug 21 23:05:24.990949442 (317h39m19.422029573s) [upload=330479]: Upload finished. err: <nil>
Duration: 2.350556432s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [............................................XX..............] duration:
5.809439878s
FAIL (2025-08-21 23:07:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.809462541s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [....X.....X.......................XX.X..............X.......] duration:
6.311910513s
FAIL (2025-08-21 23:07:38 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:01:51.826406781 (317h35m46.257486902s) [upload=330420]: Starting upload Aug 21 23:01:51.826449382 (317h35m46.257529503s) [upload=330420]: Done starting upload Aug 21 23:01:51.826636801 (317h35m46.257716932s) [upload=330420]: Uploading segments... Aug 21 23:01:51.826754176 (317h35m46.257834317s) [upload=330420]: Got next segment. Inline: false Aug 21 23:01:51.826826554 (317h35m46.257906695s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:01:51.826832295 (317h35m46.257912426s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:01:52.039215545 (317h35m46.470295676s) [upload=330420]: Next segment err: metaclient: Too Many Requests Aug 21 23:01:52.039237608 (317h35m46.470317729s) [upload=330420]: Done uploading segments... Aug 21 23:01:52.039249991 (317h35m46.470330122s) [upload=330420]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.311935881s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............................................XX.............] duration:
6.125050819s
FAIL (2025-08-21 23:07:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.125072891s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......X..................................................X..] duration:
2.002628906s
FAIL (2025-08-21 23:07:13 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 21 23:05:24.685747594 (317h39m19.116827755s) [upload=330479]: Starting upload
Aug 21 23:05:24.685873967 (317h39m19.116954098s) [upload=330479]: Done starting upload
Aug 21 23:05:24.685941395 (317h39m19.117021526s) [upload=330479]: Uploading segments...
Aug 21 23:05:24.686009706 (317h39m19.117089837s) [upload=330479]: Got next segment. Inline: true
Aug 21 23:05:24.686014836 (317h39m19.117094967s) [upload=330479]: Waiting for error group managing segments...
Aug 21 23:05:24.686047238 (317h39m19.117127359s) [upload=330479]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 23:05:24.686051566 (317h39m19.117131697s) [upload=330479]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 23:05:24.686054061 (317h39m19.117134192s) [upload=330479]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 23:05:24.990932300 (317h39m19.422012431s) [upload=330479]: Done uploading segments...
Aug 21 23:05:24.990949442 (317h39m19.422029573s) [upload=330479]: Upload finished. err: <nil>
Duration: 2.002656058s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.....X.....X.......................XX.X..............X......] duration:
6.496714119s
FAIL (2025-08-21 23:06:58 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:01:51.826406781 (317h35m46.257486902s) [upload=330420]: Starting upload Aug 21 23:01:51.826449382 (317h35m46.257529503s) [upload=330420]: Done starting upload Aug 21 23:01:51.826636801 (317h35m46.257716932s) [upload=330420]: Uploading segments... Aug 21 23:01:51.826754176 (317h35m46.257834317s) [upload=330420]: Got next segment. Inline: false Aug 21 23:01:51.826826554 (317h35m46.257906695s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:01:51.826832295 (317h35m46.257912426s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:01:52.039215545 (317h35m46.470295676s) [upload=330420]: Next segment err: metaclient: Too Many Requests Aug 21 23:01:52.039237608 (317h35m46.470317729s) [upload=330420]: Done uploading segments... Aug 21 23:01:52.039249991 (317h35m46.470330122s) [upload=330420]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.496732453s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............................................XX............] duration:
6.243383392s
FAIL (2025-08-21 23:06:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.243428318s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......X..................................................X.] duration:
2.486270377s
FAIL (2025-08-21 23:06:21 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 21 23:05:24.685747594 (317h39m19.116827755s) [upload=330479]: Starting upload
Aug 21 23:05:24.685873967 (317h39m19.116954098s) [upload=330479]: Done starting upload
Aug 21 23:05:24.685941395 (317h39m19.117021526s) [upload=330479]: Uploading segments...
Aug 21 23:05:24.686009706 (317h39m19.117089837s) [upload=330479]: Got next segment. Inline: true
Aug 21 23:05:24.686014836 (317h39m19.117094967s) [upload=330479]: Waiting for error group managing segments...
Aug 21 23:05:24.686047238 (317h39m19.117127359s) [upload=330479]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 23:05:24.686051566 (317h39m19.117131697s) [upload=330479]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 23:05:24.686054061 (317h39m19.117134192s) [upload=330479]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 23:05:24.990932300 (317h39m19.422012431s) [upload=330479]: Done uploading segments...
Aug 21 23:05:24.990949442 (317h39m19.422029573s) [upload=330479]: Upload finished. err: <nil>
Duration: 2.486298631s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [......X.....X.......................XX.X..............X.....] duration:
6.078050063s
FAIL (2025-08-21 23:06:11 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:01:51.826406781 (317h35m46.257486902s) [upload=330420]: Starting upload Aug 21 23:01:51.826449382 (317h35m46.257529503s) [upload=330420]: Done starting upload Aug 21 23:01:51.826636801 (317h35m46.257716932s) [upload=330420]: Uploading segments... Aug 21 23:01:51.826754176 (317h35m46.257834317s) [upload=330420]: Got next segment. Inline: false Aug 21 23:01:51.826826554 (317h35m46.257906695s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:01:51.826832295 (317h35m46.257912426s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:01:52.039215545 (317h35m46.470295676s) [upload=330420]: Next segment err: metaclient: Too Many Requests Aug 21 23:01:52.039237608 (317h35m46.470317729s) [upload=330420]: Done uploading segments... Aug 21 23:01:52.039249991 (317h35m46.470330122s) [upload=330420]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.078080962s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............................................XX...........] duration:
5.834855058s
FAIL (2025-08-21 23:05:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.834877893s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.......X.....X.......................XX.X..............X....] duration:
7.189545583s
FAIL (2025-08-21 23:05:31 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:01:51.826406781 (317h35m46.257486902s) [upload=330420]: Starting upload Aug 21 23:01:51.826449382 (317h35m46.257529503s) [upload=330420]: Done starting upload Aug 21 23:01:51.826636801 (317h35m46.257716932s) [upload=330420]: Uploading segments... Aug 21 23:01:51.826754176 (317h35m46.257834317s) [upload=330420]: Got next segment. Inline: false Aug 21 23:01:51.826826554 (317h35m46.257906695s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:01:51.826832295 (317h35m46.257912426s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:01:52.039215545 (317h35m46.470295676s) [upload=330420]: Next segment err: metaclient: Too Many Requests Aug 21 23:01:52.039237608 (317h35m46.470317729s) [upload=330420]: Done uploading segments... Aug 21 23:01:52.039249991 (317h35m46.470330122s) [upload=330420]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.189571582s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [........X..................................................X] duration:
1.654249934s
FAIL (2025-08-21 23:05:25 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 21 23:05:24.685747594 (317h39m19.116827755s) [upload=330479]: Starting upload
Aug 21 23:05:24.685873967 (317h39m19.116954098s) [upload=330479]: Done starting upload
Aug 21 23:05:24.685941395 (317h39m19.117021526s) [upload=330479]: Uploading segments...
Aug 21 23:05:24.686009706 (317h39m19.117089837s) [upload=330479]: Got next segment. Inline: true
Aug 21 23:05:24.686014836 (317h39m19.117094967s) [upload=330479]: Waiting for error group managing segments...
Aug 21 23:05:24.686047238 (317h39m19.117127359s) [upload=330479]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 23:05:24.686051566 (317h39m19.117131697s) [upload=330479]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 23:05:24.686054061 (317h39m19.117134192s) [upload=330479]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 23:05:24.990932300 (317h39m19.422012431s) [upload=330479]: Done uploading segments...
Aug 21 23:05:24.990949442 (317h39m19.422029573s) [upload=330479]: Upload finished. err: <nil>
Duration: 1.654285321s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [................................................XX..........] duration:
5.928335823s
FAIL (2025-08-21 23:05:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.928358547s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................................................XX.........] duration:
6.080271015s
FAIL (2025-08-21 23:04:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.08029518s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [X........X..................................................] duration:
2.644172081s
FAIL (2025-08-21 23:04:27 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b28c4ca18a201043fddca8567a02732a809c5d2fd2639c5601d57a98953650d7 != before f2cb45c3aec11d3142db99bebb64aae07bdfba077a74cc20debca35810209156 ======LOGS====== Aug 21 22:29:02.258082168 (317h2m56.689162309s) [upload=329845]: Starting upload Aug 21 22:29:02.258135651 (317h2m56.689215782s) [upload=329845]: Done starting upload Aug 21 22:29:02.258197298 (317h2m56.689277429s) [upload=329845]: Uploading segments... Aug 21 22:29:02.258242726 (317h2m56.689322857s) [upload=329845]: Got next segment. Inline: true Aug 21 22:29:02.258246483 (317h2m56.689326614s) [upload=329845]: Waiting for error group managing segments... Aug 21 22:29:02.258279716 (317h2m56.689359847s) [upload=329845]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 22:29:02.258283844 (317h2m56.689363975s) [upload=329845]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 22:29:02.258286549 (317h2m56.68936668s) [upload=329845]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 22:29:02.725381560 (317h2m57.156461691s) [upload=329845]: Done uploading segments... Aug 21 22:29:02.725398683 (317h2m57.156478814s) [upload=329845]: Upload finished. err: <nil>
Duration: 2.644200495s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
7 out of 60 is failed (threshold=3) [X.......X.....X.......................XX.X..............X...] duration:
5.970682712s
FAIL (2025-08-21 23:04:16 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:01:51.826406781 (317h35m46.257486902s) [upload=330420]: Starting upload Aug 21 23:01:51.826449382 (317h35m46.257529503s) [upload=330420]: Done starting upload Aug 21 23:01:51.826636801 (317h35m46.257716932s) [upload=330420]: Uploading segments... Aug 21 23:01:51.826754176 (317h35m46.257834317s) [upload=330420]: Got next segment. Inline: false Aug 21 23:01:51.826826554 (317h35m46.257906695s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:01:51.826832295 (317h35m46.257912426s) [upload=330420]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:01:52.039215545 (317h35m46.470295676s) [upload=330420]: Next segment err: metaclient: Too Many Requests Aug 21 23:01:52.039237608 (317h35m46.470317729s) [upload=330420]: Done uploading segments... Aug 21 23:01:52.039249991 (317h35m46.470330122s) [upload=330420]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.970705546s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.X........X.................................................] duration:
2.858179793s
FAIL (2025-08-21 23:04:04 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after b28c4ca18a201043fddca8567a02732a809c5d2fd2639c5601d57a98953650d7 != before f2cb45c3aec11d3142db99bebb64aae07bdfba077a74cc20debca35810209156 ======LOGS====== Aug 21 22:29:02.258082168 (317h2m56.689162309s) [upload=329845]: Starting upload Aug 21 22:29:02.258135651 (317h2m56.689215782s) [upload=329845]: Done starting upload Aug 21 22:29:02.258197298 (317h2m56.689277429s) [upload=329845]: Uploading segments... Aug 21 22:29:02.258242726 (317h2m56.689322857s) [upload=329845]: Got next segment. Inline: true Aug 21 22:29:02.258246483 (317h2m56.689326614s) [upload=329845]: Waiting for error group managing segments... Aug 21 22:29:02.258279716 (317h2m56.689359847s) [upload=329845]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 22:29:02.258283844 (317h2m56.689363975s) [upload=329845]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 22:29:02.258286549 (317h2m56.68936668s) [upload=329845]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 22:29:02.725381560 (317h2m57.156461691s) [upload=329845]: Done uploading segments... Aug 21 22:29:02.725398683 (317h2m57.156478814s) [upload=329845]: Upload finished. err: <nil>
Duration: 2.85821958s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|