Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
8 out of 60 is failed (threshold=3) [........X...X........XX.............X.X......X............X.] duration:
1.888506275s
FAIL (2025-08-21 23:41:32 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87129963ee18e5484901f879c007a2686ac49f04e963b08dbd3f68f163c2e642 != before 22e169f00ea97cb4ac3777b481aadca8984772682ca00b97c6a620a29a0cb99e ======LOGS====== Aug 21 23:40:43.581150952 (318h14m38.012231073s) [upload=331096]: Starting upload Aug 21 23:40:43.581240302 (318h14m38.012320433s) [upload=331096]: Done starting upload Aug 21 23:40:43.581310708 (318h14m38.012390839s) [upload=331096]: Uploading segments... Aug 21 23:40:43.581407873 (318h14m38.012488004s) [upload=331096]: Got next segment. Inline: true Aug 21 23:40:43.581413203 (318h14m38.012493334s) [upload=331096]: Waiting for error group managing segments... Aug 21 23:40:43.581466966 (318h14m38.012547097s) [upload=331096]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:40:43.581473529 (318h14m38.01255366s) [upload=331096]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:40:43.581476304 (318h14m38.012556425s) [upload=331096]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:40:44.102174001 (318h14m38.533254132s) [upload=331096]: Done uploading segments... Aug 21 23:40:44.102190403 (318h14m38.533270534s) [upload=331096]: Upload finished. err: <nil>
Duration: 1.888535832s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........X...........X.....................................] duration:
6.215190394s
FAIL (2025-08-21 23:41:21 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.215233316s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....XX......................................................] duration:
6.000974708s
FAIL (2025-08-21 23:40:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.000993043s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
8 out of 60 is failed (threshold=3) [.........X...X........XX.............X.X......X............X] duration:
1.892101205s
FAIL (2025-08-21 23:40:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 87129963ee18e5484901f879c007a2686ac49f04e963b08dbd3f68f163c2e642 != before 22e169f00ea97cb4ac3777b481aadca8984772682ca00b97c6a620a29a0cb99e ======LOGS====== Aug 21 23:40:43.581150952 (318h14m38.012231073s) [upload=331096]: Starting upload Aug 21 23:40:43.581240302 (318h14m38.012320433s) [upload=331096]: Done starting upload Aug 21 23:40:43.581310708 (318h14m38.012390839s) [upload=331096]: Uploading segments... Aug 21 23:40:43.581407873 (318h14m38.012488004s) [upload=331096]: Got next segment. Inline: true Aug 21 23:40:43.581413203 (318h14m38.012493334s) [upload=331096]: Waiting for error group managing segments... Aug 21 23:40:43.581466966 (318h14m38.012547097s) [upload=331096]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:40:43.581473529 (318h14m38.01255366s) [upload=331096]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:40:43.581476304 (318h14m38.012556425s) [upload=331096]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:40:44.102174001 (318h14m38.533254132s) [upload=331096]: Done uploading segments... Aug 21 23:40:44.102190403 (318h14m38.533270534s) [upload=331096]: Upload finished. err: <nil>
Duration: 1.89212474s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........X...........X....................................] duration:
6.057613599s
FAIL (2025-08-21 23:40:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.057659957s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [..........X...X........XX.............X.X......X............] duration:
1.954530146s
FAIL (2025-08-21 23:40:05 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 21 23:31:29.997189408 (318h5m24.428269529s) [upload=330932]: Starting upload Aug 21 23:31:29.997237992 (318h5m24.428318123s) [upload=330932]: Done starting upload Aug 21 23:31:29.997291173 (318h5m24.428371304s) [upload=330932]: Uploading segments... Aug 21 23:31:29.997331641 (318h5m24.428411772s) [upload=330932]: Got next segment. Inline: true Aug 21 23:31:29.997335839 (318h5m24.42841597s) [upload=330932]: Waiting for error group managing segments... Aug 21 23:31:29.997368992 (318h5m24.428449124s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:31:29.997374633 (318h5m24.428454764s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:31:29.997377418 (318h5m24.428457549s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:31:30.965967549 (318h5m25.39704768s) [upload=330932]: Done uploading segments... Aug 21 23:31:30.965985303 (318h5m25.397065434s) [upload=330932]: Upload finished. err: <nil>
Duration: 1.954559493s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration:
6.042023227s
FAIL (2025-08-21 23:39:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.042042654s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [............X...........X...................................] duration:
5.806203162s
FAIL (2025-08-21 23:39:33 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.806230794s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......XX....................................................] duration:
6.180963286s
FAIL (2025-08-21 23:39:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.180985939s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [...........X...X........XX.............X.X......X...........] duration:
2.036714718s
FAIL (2025-08-21 23:38:56 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 21 23:31:29.997189408 (318h5m24.428269529s) [upload=330932]: Starting upload Aug 21 23:31:29.997237992 (318h5m24.428318123s) [upload=330932]: Done starting upload Aug 21 23:31:29.997291173 (318h5m24.428371304s) [upload=330932]: Uploading segments... Aug 21 23:31:29.997331641 (318h5m24.428411772s) [upload=330932]: Got next segment. Inline: true Aug 21 23:31:29.997335839 (318h5m24.42841597s) [upload=330932]: Waiting for error group managing segments... Aug 21 23:31:29.997368992 (318h5m24.428449124s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:31:29.997374633 (318h5m24.428454764s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:31:29.997377418 (318h5m24.428457549s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:31:30.965967549 (318h5m25.39704768s) [upload=330932]: Done uploading segments... Aug 21 23:31:30.965985303 (318h5m25.397065434s) [upload=330932]: Upload finished. err: <nil>
Duration: 2.036742221s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............X...........X..................................] duration:
6.180470446s
FAIL (2025-08-21 23:38:50 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.180497055s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [............X...X........XX.............X.X......X..........] duration:
1.968606338s
FAIL (2025-08-21 23:38:26 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 21 23:31:29.997189408 (318h5m24.428269529s) [upload=330932]: Starting upload Aug 21 23:31:29.997237992 (318h5m24.428318123s) [upload=330932]: Done starting upload Aug 21 23:31:29.997291173 (318h5m24.428371304s) [upload=330932]: Uploading segments... Aug 21 23:31:29.997331641 (318h5m24.428411772s) [upload=330932]: Got next segment. Inline: true Aug 21 23:31:29.997335839 (318h5m24.42841597s) [upload=330932]: Waiting for error group managing segments... Aug 21 23:31:29.997368992 (318h5m24.428449124s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:31:29.997374633 (318h5m24.428454764s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:31:29.997377418 (318h5m24.428457549s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:31:30.965967549 (318h5m25.39704768s) [upload=330932]: Done uploading segments... Aug 21 23:31:30.965985303 (318h5m25.397065434s) [upload=330932]: Upload finished. err: <nil>
Duration: 1.968631487s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......XX...................................................] duration:
6.021178446s
FAIL (2025-08-21 23:38:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.021189536s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............X...........X.................................] duration:
5.739864462s
FAIL (2025-08-21 23:37:51 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.739891785s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [.............X...X........XX.............X.X......X.........] duration:
1.924017014s
FAIL (2025-08-21 23:37:15 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 21 23:31:29.997189408 (318h5m24.428269529s) [upload=330932]: Starting upload Aug 21 23:31:29.997237992 (318h5m24.428318123s) [upload=330932]: Done starting upload Aug 21 23:31:29.997291173 (318h5m24.428371304s) [upload=330932]: Uploading segments... Aug 21 23:31:29.997331641 (318h5m24.428411772s) [upload=330932]: Got next segment. Inline: true Aug 21 23:31:29.997335839 (318h5m24.42841597s) [upload=330932]: Waiting for error group managing segments... Aug 21 23:31:29.997368992 (318h5m24.428449124s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:31:29.997374633 (318h5m24.428454764s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:31:29.997377418 (318h5m24.428457549s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:31:30.965967549 (318h5m25.39704768s) [upload=330932]: Done uploading segments... Aug 21 23:31:30.965985303 (318h5m25.397065434s) [upload=330932]: Upload finished. err: <nil>
Duration: 1.924040348s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [..............X...X........XX.............X.X......X........] duration:
2.112761243s
FAIL (2025-08-21 23:36:59 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 21 23:31:29.997189408 (318h5m24.428269529s) [upload=330932]: Starting upload Aug 21 23:31:29.997237992 (318h5m24.428318123s) [upload=330932]: Done starting upload Aug 21 23:31:29.997291173 (318h5m24.428371304s) [upload=330932]: Uploading segments... Aug 21 23:31:29.997331641 (318h5m24.428411772s) [upload=330932]: Got next segment. Inline: true Aug 21 23:31:29.997335839 (318h5m24.42841597s) [upload=330932]: Waiting for error group managing segments... Aug 21 23:31:29.997368992 (318h5m24.428449124s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:31:29.997374633 (318h5m24.428454764s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:31:29.997377418 (318h5m24.428457549s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:31:30.965967549 (318h5m25.39704768s) [upload=330932]: Done uploading segments... Aug 21 23:31:30.965985303 (318h5m25.397065434s) [upload=330932]: Upload finished. err: <nil>
Duration: 2.11278524s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [X..............X...........X................................] duration:
6.105899419s
FAIL (2025-08-21 23:36:58 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.105928014s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........XX..................................................] duration:
6.024119644s
FAIL (2025-08-21 23:36:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.024142688s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.X..............X...........X...............................] duration:
6.200077417s
FAIL (2025-08-21 23:36:27 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.200127634s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [...............X...X........XX.............X.X......X.......] duration:
1.962074321s
FAIL (2025-08-21 23:36:21 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 21 23:31:29.997189408 (318h5m24.428269529s) [upload=330932]: Starting upload Aug 21 23:31:29.997237992 (318h5m24.428318123s) [upload=330932]: Done starting upload Aug 21 23:31:29.997291173 (318h5m24.428371304s) [upload=330932]: Uploading segments... Aug 21 23:31:29.997331641 (318h5m24.428411772s) [upload=330932]: Got next segment. Inline: true Aug 21 23:31:29.997335839 (318h5m24.42841597s) [upload=330932]: Waiting for error group managing segments... Aug 21 23:31:29.997368992 (318h5m24.428449124s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:31:29.997374633 (318h5m24.428454764s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:31:29.997377418 (318h5m24.428457549s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:31:30.965967549 (318h5m25.39704768s) [upload=330932]: Done uploading segments... Aug 21 23:31:30.965985303 (318h5m25.397065434s) [upload=330932]: Upload finished. err: <nil>
Duration: 1.962099599s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........XX.................................................] duration:
5.855633395s
FAIL (2025-08-21 23:35:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.855655997s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [................X...X........XX.............X.X......X......] duration:
1.996600183s
FAIL (2025-08-21 23:35:34 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 21 23:31:29.997189408 (318h5m24.428269529s) [upload=330932]: Starting upload Aug 21 23:31:29.997237992 (318h5m24.428318123s) [upload=330932]: Done starting upload Aug 21 23:31:29.997291173 (318h5m24.428371304s) [upload=330932]: Uploading segments... Aug 21 23:31:29.997331641 (318h5m24.428411772s) [upload=330932]: Got next segment. Inline: true Aug 21 23:31:29.997335839 (318h5m24.42841597s) [upload=330932]: Waiting for error group managing segments... Aug 21 23:31:29.997368992 (318h5m24.428449124s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:31:29.997374633 (318h5m24.428454764s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:31:29.997377418 (318h5m24.428457549s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:31:30.965967549 (318h5m25.39704768s) [upload=330932]: Done uploading segments... Aug 21 23:31:30.965985303 (318h5m25.397065434s) [upload=330932]: Upload finished. err: <nil>
Duration: 1.996636313s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
4 out of 60 is failed (threshold=3) [X.X..............X...........X..............................] duration:
6.276545102s
FAIL (2025-08-21 23:35:22 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.276574268s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [XX.X..............X...........X.............................] duration:
6.566843618s
FAIL (2025-08-21 23:35:03 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.566871371s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........XX................................................] duration:
6.098642302s
FAIL (2025-08-21 23:34:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.09865774s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [.................X...X........XX.............X.X......X.....] duration:
2.183301785s
FAIL (2025-08-21 23:34:43 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 21 23:31:29.997189408 (318h5m24.428269529s) [upload=330932]: Starting upload Aug 21 23:31:29.997237992 (318h5m24.428318123s) [upload=330932]: Done starting upload Aug 21 23:31:29.997291173 (318h5m24.428371304s) [upload=330932]: Uploading segments... Aug 21 23:31:29.997331641 (318h5m24.428411772s) [upload=330932]: Got next segment. Inline: true Aug 21 23:31:29.997335839 (318h5m24.42841597s) [upload=330932]: Waiting for error group managing segments... Aug 21 23:31:29.997368992 (318h5m24.428449124s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:31:29.997374633 (318h5m24.428454764s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:31:29.997377418 (318h5m24.428457549s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:31:30.965967549 (318h5m25.39704768s) [upload=330932]: Done uploading segments... Aug 21 23:31:30.965985303 (318h5m25.397065434s) [upload=330932]: Upload finished. err: <nil>
Duration: 2.18335178s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.XX.X..............X...........X............................] duration:
5.764208932s
FAIL (2025-08-21 23:34:05 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.764235733s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [..................X...X........XX.............X.X......X....] duration:
2.189627878s
FAIL (2025-08-21 23:34:01 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 21 23:31:29.997189408 (318h5m24.428269529s) [upload=330932]: Starting upload Aug 21 23:31:29.997237992 (318h5m24.428318123s) [upload=330932]: Done starting upload Aug 21 23:31:29.997291173 (318h5m24.428371304s) [upload=330932]: Uploading segments... Aug 21 23:31:29.997331641 (318h5m24.428411772s) [upload=330932]: Got next segment. Inline: true Aug 21 23:31:29.997335839 (318h5m24.42841597s) [upload=330932]: Waiting for error group managing segments... Aug 21 23:31:29.997368992 (318h5m24.428449124s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:31:29.997374633 (318h5m24.428454764s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:31:29.997377418 (318h5m24.428457549s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:31:30.965967549 (318h5m25.39704768s) [upload=330932]: Done uploading segments... Aug 21 23:31:30.965985303 (318h5m25.397065434s) [upload=330932]: Upload finished. err: <nil>
Duration: 2.189708643s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........XX...............................................] duration:
6.161310072s
FAIL (2025-08-21 23:33:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.161327084s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [..XX.X..............X...........X...........................] duration:
6.106255108s
FAIL (2025-08-21 23:33:34 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.106283083s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [...................X...X........XX.............X.X......X...] duration:
1.927860627s
FAIL (2025-08-21 23:33:25 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 21 23:31:29.997189408 (318h5m24.428269529s) [upload=330932]: Starting upload Aug 21 23:31:29.997237992 (318h5m24.428318123s) [upload=330932]: Done starting upload Aug 21 23:31:29.997291173 (318h5m24.428371304s) [upload=330932]: Uploading segments... Aug 21 23:31:29.997331641 (318h5m24.428411772s) [upload=330932]: Got next segment. Inline: true Aug 21 23:31:29.997335839 (318h5m24.42841597s) [upload=330932]: Waiting for error group managing segments... Aug 21 23:31:29.997368992 (318h5m24.428449124s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:31:29.997374633 (318h5m24.428454764s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:31:29.997377418 (318h5m24.428457549s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:31:30.965967549 (318h5m25.39704768s) [upload=330932]: Done uploading segments... Aug 21 23:31:30.965985303 (318h5m25.397065434s) [upload=330932]: Upload finished. err: <nil>
Duration: 1.927885264s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [...XX.X..............X...........X..........................] duration:
6.21278336s
FAIL (2025-08-21 23:33:14 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.212813338s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [............XX..............................................] duration:
6.01397863s
FAIL (2025-08-21 23:33:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.014001405s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [....................X...X........XX.............X.X......X..] duration:
2.061080218s
FAIL (2025-08-21 23:32:43 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 21 23:31:29.997189408 (318h5m24.428269529s) [upload=330932]: Starting upload Aug 21 23:31:29.997237992 (318h5m24.428318123s) [upload=330932]: Done starting upload Aug 21 23:31:29.997291173 (318h5m24.428371304s) [upload=330932]: Uploading segments... Aug 21 23:31:29.997331641 (318h5m24.428411772s) [upload=330932]: Got next segment. Inline: true Aug 21 23:31:29.997335839 (318h5m24.42841597s) [upload=330932]: Waiting for error group managing segments... Aug 21 23:31:29.997368992 (318h5m24.428449124s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:31:29.997374633 (318h5m24.428454764s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:31:29.997377418 (318h5m24.428457549s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:31:30.965967549 (318h5m25.39704768s) [upload=330932]: Done uploading segments... Aug 21 23:31:30.965985303 (318h5m25.397065434s) [upload=330932]: Upload finished. err: <nil>
Duration: 2.061107391s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [.....................X...X........XX.............X.X......X.] duration:
1.852335988s
FAIL (2025-08-21 23:32:25 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 21 23:31:29.997189408 (318h5m24.428269529s) [upload=330932]: Starting upload Aug 21 23:31:29.997237992 (318h5m24.428318123s) [upload=330932]: Done starting upload Aug 21 23:31:29.997291173 (318h5m24.428371304s) [upload=330932]: Uploading segments... Aug 21 23:31:29.997331641 (318h5m24.428411772s) [upload=330932]: Got next segment. Inline: true Aug 21 23:31:29.997335839 (318h5m24.42841597s) [upload=330932]: Waiting for error group managing segments... Aug 21 23:31:29.997368992 (318h5m24.428449124s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:31:29.997374633 (318h5m24.428454764s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:31:29.997377418 (318h5m24.428457549s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:31:30.965967549 (318h5m25.39704768s) [upload=330932]: Done uploading segments... Aug 21 23:31:30.965985303 (318h5m25.397065434s) [upload=330932]: Upload finished. err: <nil>
Duration: 1.852357851s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [....XX.X..............X...........X.........................] duration:
6.153795493s
FAIL (2025-08-21 23:32:23 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.153840109s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............XX.............................................] duration:
6.241450883s
FAIL (2025-08-21 23:31:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.241472273s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.....XX.X..............X...........X........................] duration:
5.906151522s
FAIL (2025-08-21 23:31:46 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.906169366s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [......................X...X........XX.............X.X......X] duration:
2.102229522s
FAIL (2025-08-21 23:31:31 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 21 23:31:29.997189408 (318h5m24.428269529s) [upload=330932]: Starting upload Aug 21 23:31:29.997237992 (318h5m24.428318123s) [upload=330932]: Done starting upload Aug 21 23:31:29.997291173 (318h5m24.428371304s) [upload=330932]: Uploading segments... Aug 21 23:31:29.997331641 (318h5m24.428411772s) [upload=330932]: Got next segment. Inline: true Aug 21 23:31:29.997335839 (318h5m24.42841597s) [upload=330932]: Waiting for error group managing segments... Aug 21 23:31:29.997368992 (318h5m24.428449124s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:31:29.997374633 (318h5m24.428454764s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:31:29.997377418 (318h5m24.428457549s) [upload=330932]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:31:30.965967549 (318h5m25.39704768s) [upload=330932]: Done uploading segments... Aug 21 23:31:30.965985303 (318h5m25.397065434s) [upload=330932]: Upload finished. err: <nil>
Duration: 2.102256864s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [......XX.X..............X...........X.......................] duration:
5.998494583s
FAIL (2025-08-21 23:31:17 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.998523989s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............XX............................................] duration:
5.930564928s
FAIL (2025-08-21 23:30:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.930588232s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
6 out of 60 is failed (threshold=3) [.......................X...X........XX.............X.X......] duration:
1.904729719s
FAIL (2025-08-21 23:30:52 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 38c9239c7be0f9590decfcd9e04ddfdedfb12e7aaf4b98990689ac8e70b4311b != before d7ffaa78fd733d6399224c0a9ee137c7b14864622c769eb900e829f3322931a6 ======LOGS====== Aug 21 23:26:17.897968945 (318h0m12.329049066s) [upload=330846]: Starting upload Aug 21 23:26:17.898004473 (318h0m12.329084604s) [upload=330846]: Done starting upload Aug 21 23:26:17.898048928 (318h0m12.329129059s) [upload=330846]: Uploading segments... Aug 21 23:26:17.898083935 (318h0m12.329164066s) [upload=330846]: Got next segment. Inline: true Aug 21 23:26:17.898087011 (318h0m12.329167142s) [upload=330846]: Waiting for error group managing segments... Aug 21 23:26:17.898116949 (318h0m12.32919708s) [upload=330846]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:26:17.898120946 (318h0m12.329201077s) [upload=330846]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:26:17.898123691 (318h0m12.329203823s) [upload=330846]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:26:18.194503530 (318h0m12.625583661s) [upload=330846]: Done uploading segments... Aug 21 23:26:18.194525361 (318h0m12.625605492s) [upload=330846]: Upload finished. err: <nil>
Duration: 1.904749898s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............XX...........................................] duration:
6.006712032s
FAIL (2025-08-21 23:30:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.006742321s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
6 out of 60 is failed (threshold=3) [........................X...X........XX.............X.X.....] duration:
2.084870558s
FAIL (2025-08-21 23:30:12 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 38c9239c7be0f9590decfcd9e04ddfdedfb12e7aaf4b98990689ac8e70b4311b != before d7ffaa78fd733d6399224c0a9ee137c7b14864622c769eb900e829f3322931a6 ======LOGS====== Aug 21 23:26:17.897968945 (318h0m12.329049066s) [upload=330846]: Starting upload Aug 21 23:26:17.898004473 (318h0m12.329084604s) [upload=330846]: Done starting upload Aug 21 23:26:17.898048928 (318h0m12.329129059s) [upload=330846]: Uploading segments... Aug 21 23:26:17.898083935 (318h0m12.329164066s) [upload=330846]: Got next segment. Inline: true Aug 21 23:26:17.898087011 (318h0m12.329167142s) [upload=330846]: Waiting for error group managing segments... Aug 21 23:26:17.898116949 (318h0m12.32919708s) [upload=330846]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:26:17.898120946 (318h0m12.329201077s) [upload=330846]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:26:17.898123691 (318h0m12.329203823s) [upload=330846]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:26:18.194503530 (318h0m12.625583661s) [upload=330846]: Done uploading segments... Aug 21 23:26:18.194525361 (318h0m12.625605492s) [upload=330846]: Upload finished. err: <nil>
Duration: 2.084906628s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......XX.X..............X...........X......................] duration:
6.115398792s
FAIL (2025-08-21 23:30:09 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.115424832s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
6 out of 60 is failed (threshold=3) [.........................X...X........XX.............X.X....] duration:
1.989233125s
FAIL (2025-08-21 23:29:52 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 38c9239c7be0f9590decfcd9e04ddfdedfb12e7aaf4b98990689ac8e70b4311b != before d7ffaa78fd733d6399224c0a9ee137c7b14864622c769eb900e829f3322931a6 ======LOGS====== Aug 21 23:26:17.897968945 (318h0m12.329049066s) [upload=330846]: Starting upload Aug 21 23:26:17.898004473 (318h0m12.329084604s) [upload=330846]: Done starting upload Aug 21 23:26:17.898048928 (318h0m12.329129059s) [upload=330846]: Uploading segments... Aug 21 23:26:17.898083935 (318h0m12.329164066s) [upload=330846]: Got next segment. Inline: true Aug 21 23:26:17.898087011 (318h0m12.329167142s) [upload=330846]: Waiting for error group managing segments... Aug 21 23:26:17.898116949 (318h0m12.32919708s) [upload=330846]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:26:17.898120946 (318h0m12.329201077s) [upload=330846]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:26:17.898123691 (318h0m12.329203823s) [upload=330846]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:26:18.194503530 (318h0m12.625583661s) [upload=330846]: Done uploading segments... Aug 21 23:26:18.194525361 (318h0m12.625605492s) [upload=330846]: Upload finished. err: <nil>
Duration: 1.989253475s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [................XX..........................................] duration:
6.195258399s
FAIL (2025-08-21 23:29:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.195280261s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [........XX.X..............X...........X.....................] duration:
6.171007993s
FAIL (2025-08-21 23:28:59 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 21 23:11:30.202122178 (317h45m24.633202309s) [upload=330588]: Starting upload Aug 21 23:11:30.202182443 (317h45m24.633262574s) [upload=330588]: Done starting upload Aug 21 23:11:30.202313304 (317h45m24.633393435s) [upload=330588]: Uploading segments... Aug 21 23:11:30.207769824 (317h45m24.638849955s) [upload=330588]: Got next segment. Inline: false Aug 21 23:11:30.207842683 (317h45m24.638922814s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:11:30.207858754 (317h45m24.638938885s) [upload=330588]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 21 23:11:30.421157246 (317h45m24.852237377s) [upload=330588]: Next segment err: metaclient: Too Many Requests Aug 21 23:11:30.421175901 (317h45m24.852256022s) [upload=330588]: Done uploading segments... Aug 21 23:11:30.421188716 (317h45m24.852268857s) [upload=330588]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.171033971s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
6 out of 60 is failed (threshold=3) [..........................X...X........XX.............X.X...] duration:
1.892948012s
FAIL (2025-08-21 23:28:53 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 38c9239c7be0f9590decfcd9e04ddfdedfb12e7aaf4b98990689ac8e70b4311b != before d7ffaa78fd733d6399224c0a9ee137c7b14864622c769eb900e829f3322931a6 ======LOGS====== Aug 21 23:26:17.897968945 (318h0m12.329049066s) [upload=330846]: Starting upload Aug 21 23:26:17.898004473 (318h0m12.329084604s) [upload=330846]: Done starting upload Aug 21 23:26:17.898048928 (318h0m12.329129059s) [upload=330846]: Uploading segments... Aug 21 23:26:17.898083935 (318h0m12.329164066s) [upload=330846]: Got next segment. Inline: true Aug 21 23:26:17.898087011 (318h0m12.329167142s) [upload=330846]: Waiting for error group managing segments... Aug 21 23:26:17.898116949 (318h0m12.32919708s) [upload=330846]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 21 23:26:17.898120946 (318h0m12.329201077s) [upload=330846]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 21 23:26:17.898123691 (318h0m12.329203823s) [upload=330846]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 21 23:26:18.194503530 (318h0m12.625583661s) [upload=330846]: Done uploading segments... Aug 21 23:26:18.194525361 (318h0m12.625605492s) [upload=330846]: Upload finished. err: <nil>
Duration: 1.89297212s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................XX.........................................] duration:
6.13639048s
FAIL (2025-08-21 23:28:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.136409437s
edge
gateway
us-select-1
|