Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
2.343181314s
FAIL (2025-08-12 09:33:25 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.343202446s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
6.572433842s
FAIL (2025-08-12 09:33:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.572454771s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [....X....................................X.......X........X.] duration:
1.835356783s
FAIL (2025-08-12 09:33:00 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after c36fafdabf96bb5b9c4464bc922fb40045dd340ad6b7f61aa3a428437d8c7f85 != before c9b346cbcc10dabc7edd608987d023da7f45c3973830ffa574bb2139ca951f3f ======LOGS====== Aug 12 09:32:43.528352715 (88h6m37.959432847s) [upload=91793]: Starting upload Aug 12 09:32:43.528398883 (88h6m37.959479014s) [upload=91793]: Done starting upload Aug 12 09:32:43.528451965 (88h6m37.959532126s) [upload=91793]: Uploading segments... Aug 12 09:32:43.528498204 (88h6m37.959578365s) [upload=91793]: Got next segment. Inline: true Aug 12 09:32:43.528504716 (88h6m37.959584867s) [upload=91793]: Waiting for error group managing segments... Aug 12 09:32:43.528567947 (88h6m37.959648078s) [upload=91793]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:32:43.528574670 (88h6m37.959654861s) [upload=91793]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 09:32:43.528579549 (88h6m37.9596597s) [upload=91793]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 09:32:43.987672872 (88h6m38.418753003s) [upload=91793]: Done uploading segments... Aug 12 09:32:43.987690746 (88h6m38.418770878s) [upload=91793]: Upload finished. err: <nil>
Duration: 1.835383143s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.....X....................................X.......X........X] duration:
1.789624296s
FAIL (2025-08-12 09:32:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after c36fafdabf96bb5b9c4464bc922fb40045dd340ad6b7f61aa3a428437d8c7f85 != before c9b346cbcc10dabc7edd608987d023da7f45c3973830ffa574bb2139ca951f3f ======LOGS====== Aug 12 09:32:43.528352715 (88h6m37.959432847s) [upload=91793]: Starting upload Aug 12 09:32:43.528398883 (88h6m37.959479014s) [upload=91793]: Done starting upload Aug 12 09:32:43.528451965 (88h6m37.959532126s) [upload=91793]: Uploading segments... Aug 12 09:32:43.528498204 (88h6m37.959578365s) [upload=91793]: Got next segment. Inline: true Aug 12 09:32:43.528504716 (88h6m37.959584867s) [upload=91793]: Waiting for error group managing segments... Aug 12 09:32:43.528567947 (88h6m37.959648078s) [upload=91793]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:32:43.528574670 (88h6m37.959654861s) [upload=91793]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 09:32:43.528579549 (88h6m37.9596597s) [upload=91793]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 09:32:43.987672872 (88h6m38.418753003s) [upload=91793]: Done uploading segments... Aug 12 09:32:43.987690746 (88h6m38.418770878s) [upload=91793]: Upload finished. err: <nil>
Duration: 1.789676396s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [....X........X......X......................X....X...........] duration:
5.780335963s
FAIL (2025-08-12 09:32:20 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 09:23:08.516266754 (87h57m2.947346885s) [upload=91621]: Starting upload Aug 12 09:23:08.516319585 (87h57m2.947399716s) [upload=91621]: Done starting upload Aug 12 09:23:08.516447219 (87h57m2.947527371s) [upload=91621]: Uploading segments... Aug 12 09:23:08.516526641 (87h57m2.947606772s) [upload=91621]: Got next segment. Inline: false Aug 12 09:23:08.516564794 (87h57m2.947644925s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:23:08.516570595 (87h57m2.947650736s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 12 09:23:08.760094669 (87h57m3.1911748s) [upload=91621]: Next segment err: metaclient: Too Many Requests Aug 12 09:23:08.760114537 (87h57m3.191194668s) [upload=91621]: Done uploading segments... Aug 12 09:23:08.760127261 (87h57m3.191207392s) [upload=91621]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.78036017s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [......X....................................X.......X........] duration:
1.859524779s
FAIL (2025-08-12 09:32:16 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 09:26:03.373412323 (87h59m57.804492454s) [upload=91674]: Starting upload Aug 12 09:26:03.373459974 (87h59m57.804540105s) [upload=91674]: Done starting upload Aug 12 09:26:03.373513897 (87h59m57.804594028s) [upload=91674]: Uploading segments... Aug 12 09:26:03.373573772 (87h59m57.804653903s) [upload=91674]: Got next segment. Inline: true Aug 12 09:26:03.373579032 (87h59m57.804659163s) [upload=91674]: Waiting for error group managing segments... Aug 12 09:26:03.373615531 (87h59m57.804695662s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:26:03.373620261 (87h59m57.804700392s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 09:26:03.373622966 (87h59m57.804703087s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 09:26:03.791789064 (87h59m58.222869195s) [upload=91674]: Done uploading segments... Aug 12 09:26:03.791806668 (87h59m58.222886799s) [upload=91674]: Upload finished. err: <nil>
Duration: 1.859550068s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.323893274s
FAIL (2025-08-12 09:32:14 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.323930135s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
6.457771775s
FAIL (2025-08-12 09:31:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.457792305s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.......X....................................X.......X.......] duration:
1.824544285s
FAIL (2025-08-12 09:31:45 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 09:26:03.373412323 (87h59m57.804492454s) [upload=91674]: Starting upload Aug 12 09:26:03.373459974 (87h59m57.804540105s) [upload=91674]: Done starting upload Aug 12 09:26:03.373513897 (87h59m57.804594028s) [upload=91674]: Uploading segments... Aug 12 09:26:03.373573772 (87h59m57.804653903s) [upload=91674]: Got next segment. Inline: true Aug 12 09:26:03.373579032 (87h59m57.804659163s) [upload=91674]: Waiting for error group managing segments... Aug 12 09:26:03.373615531 (87h59m57.804695662s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:26:03.373620261 (87h59m57.804700392s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 09:26:03.373622966 (87h59m57.804703087s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 09:26:03.791789064 (87h59m58.222869195s) [upload=91674]: Done uploading segments... Aug 12 09:26:03.791806668 (87h59m58.222886799s) [upload=91674]: Upload finished. err: <nil>
Duration: 1.824591724s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.....X........X......X......................X....X..........] duration:
5.817501757s
FAIL (2025-08-12 09:31:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 09:23:08.516266754 (87h57m2.947346885s) [upload=91621]: Starting upload Aug 12 09:23:08.516319585 (87h57m2.947399716s) [upload=91621]: Done starting upload Aug 12 09:23:08.516447219 (87h57m2.947527371s) [upload=91621]: Uploading segments... Aug 12 09:23:08.516526641 (87h57m2.947606772s) [upload=91621]: Got next segment. Inline: false Aug 12 09:23:08.516564794 (87h57m2.947644925s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:23:08.516570595 (87h57m2.947650736s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 12 09:23:08.760094669 (87h57m3.1911748s) [upload=91621]: Next segment err: metaclient: Too Many Requests Aug 12 09:23:08.760114537 (87h57m3.191194668s) [upload=91621]: Done uploading segments... Aug 12 09:23:08.760127261 (87h57m3.191207392s) [upload=91621]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.817532635s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
6.169623521s
FAIL (2025-08-12 09:31:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.169648649s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [........X....................................X.......X......] duration:
1.860013497s
FAIL (2025-08-12 09:31:10 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 09:26:03.373412323 (87h59m57.804492454s) [upload=91674]: Starting upload Aug 12 09:26:03.373459974 (87h59m57.804540105s) [upload=91674]: Done starting upload Aug 12 09:26:03.373513897 (87h59m57.804594028s) [upload=91674]: Uploading segments... Aug 12 09:26:03.373573772 (87h59m57.804653903s) [upload=91674]: Got next segment. Inline: true Aug 12 09:26:03.373579032 (87h59m57.804659163s) [upload=91674]: Waiting for error group managing segments... Aug 12 09:26:03.373615531 (87h59m57.804695662s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:26:03.373620261 (87h59m57.804700392s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 09:26:03.373622966 (87h59m57.804703087s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 09:26:03.791789064 (87h59m58.222869195s) [upload=91674]: Done uploading segments... Aug 12 09:26:03.791806668 (87h59m58.222886799s) [upload=91674]: Upload finished. err: <nil>
Duration: 1.860039265s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.463089625s
FAIL (2025-08-12 09:31:08 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.463113941s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [......X........X......X......................X....X.........] duration:
5.776023439s
FAIL (2025-08-12 09:31:02 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 09:23:08.516266754 (87h57m2.947346885s) [upload=91621]: Starting upload Aug 12 09:23:08.516319585 (87h57m2.947399716s) [upload=91621]: Done starting upload Aug 12 09:23:08.516447219 (87h57m2.947527371s) [upload=91621]: Uploading segments... Aug 12 09:23:08.516526641 (87h57m2.947606772s) [upload=91621]: Got next segment. Inline: false Aug 12 09:23:08.516564794 (87h57m2.947644925s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:23:08.516570595 (87h57m2.947650736s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 12 09:23:08.760094669 (87h57m3.1911748s) [upload=91621]: Next segment err: metaclient: Too Many Requests Aug 12 09:23:08.760114537 (87h57m3.191194668s) [upload=91621]: Done uploading segments... Aug 12 09:23:08.760127261 (87h57m3.191207392s) [upload=91621]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.776051032s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
6.264138207s
FAIL (2025-08-12 09:30:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.26416096s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.490223807s
FAIL (2025-08-12 09:30:25 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.490245119s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.........X....................................X.......X.....] duration:
1.87529867s
FAIL (2025-08-12 09:29:57 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 09:26:03.373412323 (87h59m57.804492454s) [upload=91674]: Starting upload Aug 12 09:26:03.373459974 (87h59m57.804540105s) [upload=91674]: Done starting upload Aug 12 09:26:03.373513897 (87h59m57.804594028s) [upload=91674]: Uploading segments... Aug 12 09:26:03.373573772 (87h59m57.804653903s) [upload=91674]: Got next segment. Inline: true Aug 12 09:26:03.373579032 (87h59m57.804659163s) [upload=91674]: Waiting for error group managing segments... Aug 12 09:26:03.373615531 (87h59m57.804695662s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:26:03.373620261 (87h59m57.804700392s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 09:26:03.373622966 (87h59m57.804703087s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 09:26:03.791789064 (87h59m58.222869195s) [upload=91674]: Done uploading segments... Aug 12 09:26:03.791806668 (87h59m58.222886799s) [upload=91674]: Upload finished. err: <nil>
Duration: 1.875324931s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......X........X......X......................X....X........] duration:
5.557345005s
FAIL (2025-08-12 09:29:56 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 09:23:08.516266754 (87h57m2.947346885s) [upload=91621]: Starting upload Aug 12 09:23:08.516319585 (87h57m2.947399716s) [upload=91621]: Done starting upload Aug 12 09:23:08.516447219 (87h57m2.947527371s) [upload=91621]: Uploading segments... Aug 12 09:23:08.516526641 (87h57m2.947606772s) [upload=91621]: Got next segment. Inline: false Aug 12 09:23:08.516564794 (87h57m2.947644925s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:23:08.516570595 (87h57m2.947650736s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 12 09:23:08.760094669 (87h57m3.1911748s) [upload=91621]: Next segment err: metaclient: Too Many Requests Aug 12 09:23:08.760114537 (87h57m3.191194668s) [upload=91621]: Done uploading segments... Aug 12 09:23:08.760127261 (87h57m3.191207392s) [upload=91621]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.557377447s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
5.903058514s
FAIL (2025-08-12 09:29:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.903089013s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.283966013s
FAIL (2025-08-12 09:29:25 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.283987395s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..........X....................................X.......X....] duration:
1.814374917s
FAIL (2025-08-12 09:29:12 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 09:26:03.373412323 (87h59m57.804492454s) [upload=91674]: Starting upload Aug 12 09:26:03.373459974 (87h59m57.804540105s) [upload=91674]: Done starting upload Aug 12 09:26:03.373513897 (87h59m57.804594028s) [upload=91674]: Uploading segments... Aug 12 09:26:03.373573772 (87h59m57.804653903s) [upload=91674]: Got next segment. Inline: true Aug 12 09:26:03.373579032 (87h59m57.804659163s) [upload=91674]: Waiting for error group managing segments... Aug 12 09:26:03.373615531 (87h59m57.804695662s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:26:03.373620261 (87h59m57.804700392s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 09:26:03.373622966 (87h59m57.804703087s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 09:26:03.791789064 (87h59m58.222869195s) [upload=91674]: Done uploading segments... Aug 12 09:26:03.791806668 (87h59m58.222886799s) [upload=91674]: Upload finished. err: <nil>
Duration: 1.814401858s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
6.072214983s
FAIL (2025-08-12 09:29:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.072256172s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.311786803s
FAIL (2025-08-12 09:28:51 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.311806881s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [........X........X......X......................X....X.......] duration:
5.699117405s
FAIL (2025-08-12 09:28:48 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 09:23:08.516266754 (87h57m2.947346885s) [upload=91621]: Starting upload Aug 12 09:23:08.516319585 (87h57m2.947399716s) [upload=91621]: Done starting upload Aug 12 09:23:08.516447219 (87h57m2.947527371s) [upload=91621]: Uploading segments... Aug 12 09:23:08.516526641 (87h57m2.947606772s) [upload=91621]: Got next segment. Inline: false Aug 12 09:23:08.516564794 (87h57m2.947644925s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:23:08.516570595 (87h57m2.947650736s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 12 09:23:08.760094669 (87h57m3.1911748s) [upload=91621]: Next segment err: metaclient: Too Many Requests Aug 12 09:23:08.760114537 (87h57m3.191194668s) [upload=91621]: Done uploading segments... Aug 12 09:23:08.760127261 (87h57m3.191207392s) [upload=91621]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.699143935s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.390877368s
FAIL (2025-08-12 09:28:22 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.390905871s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [...........X....................................X.......X...] duration:
1.833323651s
FAIL (2025-08-12 09:28:07 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 09:26:03.373412323 (87h59m57.804492454s) [upload=91674]: Starting upload Aug 12 09:26:03.373459974 (87h59m57.804540105s) [upload=91674]: Done starting upload Aug 12 09:26:03.373513897 (87h59m57.804594028s) [upload=91674]: Uploading segments... Aug 12 09:26:03.373573772 (87h59m57.804653903s) [upload=91674]: Got next segment. Inline: true Aug 12 09:26:03.373579032 (87h59m57.804659163s) [upload=91674]: Waiting for error group managing segments... Aug 12 09:26:03.373615531 (87h59m57.804695662s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:26:03.373620261 (87h59m57.804700392s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 09:26:03.373622966 (87h59m57.804703087s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 09:26:03.791789064 (87h59m58.222869195s) [upload=91674]: Done uploading segments... Aug 12 09:26:03.791806668 (87h59m58.222886799s) [upload=91674]: Upload finished. err: <nil>
Duration: 1.83334895s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
6.783279281s
FAIL (2025-08-12 09:28:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.783293357s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.762086978s
FAIL (2025-08-12 09:28:03 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.762112487s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [.........X........X......X......................X....X......] duration:
5.546170232s
FAIL (2025-08-12 09:27:57 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 09:23:08.516266754 (87h57m2.947346885s) [upload=91621]: Starting upload Aug 12 09:23:08.516319585 (87h57m2.947399716s) [upload=91621]: Done starting upload Aug 12 09:23:08.516447219 (87h57m2.947527371s) [upload=91621]: Uploading segments... Aug 12 09:23:08.516526641 (87h57m2.947606772s) [upload=91621]: Got next segment. Inline: false Aug 12 09:23:08.516564794 (87h57m2.947644925s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:23:08.516570595 (87h57m2.947650736s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 12 09:23:08.760094669 (87h57m3.1911748s) [upload=91621]: Next segment err: metaclient: Too Many Requests Aug 12 09:23:08.760114537 (87h57m3.191194668s) [upload=91621]: Done uploading segments... Aug 12 09:23:08.760127261 (87h57m3.191207392s) [upload=91621]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.54619957s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [............X....................................X.......X..] duration:
1.979221325s
FAIL (2025-08-12 09:27:33 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 09:26:03.373412323 (87h59m57.804492454s) [upload=91674]: Starting upload Aug 12 09:26:03.373459974 (87h59m57.804540105s) [upload=91674]: Done starting upload Aug 12 09:26:03.373513897 (87h59m57.804594028s) [upload=91674]: Uploading segments... Aug 12 09:26:03.373573772 (87h59m57.804653903s) [upload=91674]: Got next segment. Inline: true Aug 12 09:26:03.373579032 (87h59m57.804659163s) [upload=91674]: Waiting for error group managing segments... Aug 12 09:26:03.373615531 (87h59m57.804695662s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:26:03.373620261 (87h59m57.804700392s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 09:26:03.373622966 (87h59m57.804703087s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 09:26:03.791789064 (87h59m58.222869195s) [upload=91674]: Done uploading segments... Aug 12 09:26:03.791806668 (87h59m58.222886799s) [upload=91674]: Upload finished. err: <nil>
Duration: 1.979246543s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.349844013s
FAIL (2025-08-12 09:27:30 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.34987374s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
5 out of 60 is failed (threshold=3) [..........X........X......X......................X....X.....] duration:
5.894176585s
FAIL (2025-08-12 09:27:09 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 09:23:08.516266754 (87h57m2.947346885s) [upload=91621]: Starting upload Aug 12 09:23:08.516319585 (87h57m2.947399716s) [upload=91621]: Done starting upload Aug 12 09:23:08.516447219 (87h57m2.947527371s) [upload=91621]: Uploading segments... Aug 12 09:23:08.516526641 (87h57m2.947606772s) [upload=91621]: Got next segment. Inline: false Aug 12 09:23:08.516564794 (87h57m2.947644925s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:23:08.516570595 (87h57m2.947650736s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 12 09:23:08.760094669 (87h57m3.1911748s) [upload=91621]: Next segment err: metaclient: Too Many Requests Aug 12 09:23:08.760114537 (87h57m3.191194668s) [upload=91621]: Done uploading segments... Aug 12 09:23:08.760127261 (87h57m3.191207392s) [upload=91621]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.894201932s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
6.238089483s
FAIL (2025-08-12 09:27:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.238110443s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.48226915s
FAIL (2025-08-12 09:26:48 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.482289909s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
5.914017907s
FAIL (2025-08-12 09:26:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.914041553s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.............X....................................X.......X.] duration:
1.836032044s
FAIL (2025-08-12 09:26:31 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 09:26:03.373412323 (87h59m57.804492454s) [upload=91674]: Starting upload Aug 12 09:26:03.373459974 (87h59m57.804540105s) [upload=91674]: Done starting upload Aug 12 09:26:03.373513897 (87h59m57.804594028s) [upload=91674]: Uploading segments... Aug 12 09:26:03.373573772 (87h59m57.804653903s) [upload=91674]: Got next segment. Inline: true Aug 12 09:26:03.373579032 (87h59m57.804659163s) [upload=91674]: Waiting for error group managing segments... Aug 12 09:26:03.373615531 (87h59m57.804695662s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:26:03.373620261 (87h59m57.804700392s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 09:26:03.373622966 (87h59m57.804703087s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 09:26:03.791789064 (87h59m58.222869195s) [upload=91674]: Done uploading segments... Aug 12 09:26:03.791806668 (87h59m58.222886799s) [upload=91674]: Upload finished. err: <nil>
Duration: 1.836063155s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
6.459162871s
FAIL (2025-08-12 09:26:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.459184312s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
5.970040043s
FAIL (2025-08-12 09:26:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.970061773s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [X..........X........X......X......................X....X....] duration:
5.70238411s
FAIL (2025-08-12 09:26:06 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 09:23:08.516266754 (87h57m2.947346885s) [upload=91621]: Starting upload Aug 12 09:23:08.516319585 (87h57m2.947399716s) [upload=91621]: Done starting upload Aug 12 09:23:08.516447219 (87h57m2.947527371s) [upload=91621]: Uploading segments... Aug 12 09:23:08.516526641 (87h57m2.947606772s) [upload=91621]: Got next segment. Inline: false Aug 12 09:23:08.516564794 (87h57m2.947644925s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:23:08.516570595 (87h57m2.947650736s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 12 09:23:08.760094669 (87h57m3.1911748s) [upload=91621]: Next segment err: metaclient: Too Many Requests Aug 12 09:23:08.760114537 (87h57m3.191194668s) [upload=91621]: Done uploading segments... Aug 12 09:23:08.760127261 (87h57m3.191207392s) [upload=91621]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.702415691s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [X.............X....................................X.......X] duration:
1.507799736s
FAIL (2025-08-12 09:26:04 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 12 09:26:03.373412323 (87h59m57.804492454s) [upload=91674]: Starting upload Aug 12 09:26:03.373459974 (87h59m57.804540105s) [upload=91674]: Done starting upload Aug 12 09:26:03.373513897 (87h59m57.804594028s) [upload=91674]: Uploading segments... Aug 12 09:26:03.373573772 (87h59m57.804653903s) [upload=91674]: Got next segment. Inline: true Aug 12 09:26:03.373579032 (87h59m57.804659163s) [upload=91674]: Waiting for error group managing segments... Aug 12 09:26:03.373615531 (87h59m57.804695662s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:26:03.373620261 (87h59m57.804700392s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 09:26:03.373622966 (87h59m57.804703087s) [upload=91674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 09:26:03.791789064 (87h59m58.222869195s) [upload=91674]: Done uploading segments... Aug 12 09:26:03.791806668 (87h59m58.222886799s) [upload=91674]: Upload finished. err: <nil>
Duration: 1.507842888s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.633558069s
FAIL (2025-08-12 09:25:50 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.633668801s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
6.723826044s
FAIL (2025-08-12 09:25:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.723849929s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [.X..........X........X......X......................X....X...] duration:
5.703005359s
FAIL (2025-08-12 09:25:19 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 09:23:08.516266754 (87h57m2.947346885s) [upload=91621]: Starting upload Aug 12 09:23:08.516319585 (87h57m2.947399716s) [upload=91621]: Done starting upload Aug 12 09:23:08.516447219 (87h57m2.947527371s) [upload=91621]: Uploading segments... Aug 12 09:23:08.516526641 (87h57m2.947606772s) [upload=91621]: Got next segment. Inline: false Aug 12 09:23:08.516564794 (87h57m2.947644925s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:23:08.516570595 (87h57m2.947650736s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 12 09:23:08.760094669 (87h57m3.1911748s) [upload=91621]: Next segment err: metaclient: Too Many Requests Aug 12 09:23:08.760114537 (87h57m3.191194668s) [upload=91621]: Done uploading segments... Aug 12 09:23:08.760127261 (87h57m3.191207392s) [upload=91621]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.703030408s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [.X.............X....................................X.......] duration:
1.922091813s
FAIL (2025-08-12 09:25:18 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after e0862886d4701b2ffa37c8b96ae06e66a26fee0264d798fd8ddd05cd6756d482 != before 67eae7018ec0721911b083a6cc56a1db166e830f16615ea2aa349b10efff7b96 ======LOGS====== Aug 12 09:18:47.995749386 (87h52m42.426829517s) [upload=91548]: Starting upload Aug 12 09:18:47.995793641 (87h52m42.426873772s) [upload=91548]: Done starting upload Aug 12 09:18:47.995849598 (87h52m42.426929729s) [upload=91548]: Uploading segments... Aug 12 09:18:47.995931875 (87h52m42.427012006s) [upload=91548]: Got next segment. Inline: true Aug 12 09:18:47.995937776 (87h52m42.427017907s) [upload=91548]: Waiting for error group managing segments... Aug 12 09:18:47.995977833 (87h52m42.427057964s) [upload=91548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:18:47.995982212 (87h52m42.427062343s) [upload=91548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 09:18:47.995985047 (87h52m42.427065178s) [upload=91548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 09:18:48.717866175 (87h52m43.148946307s) [upload=91548]: Done uploading segments... Aug 12 09:18:48.717884761 (87h52m43.148964882s) [upload=91548]: Upload finished. err: <nil>
Duration: 1.922115969s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.840265341s
FAIL (2025-08-12 09:25:11 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.840290129s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [..X..........X........X......X......................X....X..] duration:
5.675011404s
FAIL (2025-08-12 09:24:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 09:23:08.516266754 (87h57m2.947346885s) [upload=91621]: Starting upload Aug 12 09:23:08.516319585 (87h57m2.947399716s) [upload=91621]: Done starting upload Aug 12 09:23:08.516447219 (87h57m2.947527371s) [upload=91621]: Uploading segments... Aug 12 09:23:08.516526641 (87h57m2.947606772s) [upload=91621]: Got next segment. Inline: false Aug 12 09:23:08.516564794 (87h57m2.947644925s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:23:08.516570595 (87h57m2.947650736s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 12 09:23:08.760094669 (87h57m3.1911748s) [upload=91621]: Next segment err: metaclient: Too Many Requests Aug 12 09:23:08.760114537 (87h57m3.191194668s) [upload=91621]: Done uploading segments... Aug 12 09:23:08.760127261 (87h57m3.191207392s) [upload=91621]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.675037804s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
6.301774991s
FAIL (2025-08-12 09:24:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.301802013s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
6 out of 60 is failed (threshold=3) [...X..........X........X......X......................X....X.] duration:
5.612820717s
FAIL (2025-08-12 09:24:08 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 12 09:23:08.516266754 (87h57m2.947346885s) [upload=91621]: Starting upload Aug 12 09:23:08.516319585 (87h57m2.947399716s) [upload=91621]: Done starting upload Aug 12 09:23:08.516447219 (87h57m2.947527371s) [upload=91621]: Uploading segments... Aug 12 09:23:08.516526641 (87h57m2.947606772s) [upload=91621]: Got next segment. Inline: false Aug 12 09:23:08.516564794 (87h57m2.947644925s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:23:08.516570595 (87h57m2.947650736s) [upload=91621]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 12 09:23:08.760094669 (87h57m3.1911748s) [upload=91621]: Next segment err: metaclient: Too Many Requests Aug 12 09:23:08.760114537 (87h57m3.191194668s) [upload=91621]: Done uploading segments... Aug 12 09:23:08.760127261 (87h57m3.191207392s) [upload=91621]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.612849854s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [..X.............X....................................X......] duration:
1.86809608s
FAIL (2025-08-12 09:24:06 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after e0862886d4701b2ffa37c8b96ae06e66a26fee0264d798fd8ddd05cd6756d482 != before 67eae7018ec0721911b083a6cc56a1db166e830f16615ea2aa349b10efff7b96 ======LOGS====== Aug 12 09:18:47.995749386 (87h52m42.426829517s) [upload=91548]: Starting upload Aug 12 09:18:47.995793641 (87h52m42.426873772s) [upload=91548]: Done starting upload Aug 12 09:18:47.995849598 (87h52m42.426929729s) [upload=91548]: Uploading segments... Aug 12 09:18:47.995931875 (87h52m42.427012006s) [upload=91548]: Got next segment. Inline: true Aug 12 09:18:47.995937776 (87h52m42.427017907s) [upload=91548]: Waiting for error group managing segments... Aug 12 09:18:47.995977833 (87h52m42.427057964s) [upload=91548]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 12 09:18:47.995982212 (87h52m42.427062343s) [upload=91548]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 12 09:18:47.995985047 (87h52m42.427065178s) [upload=91548]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 12 09:18:48.717866175 (87h52m43.148946307s) [upload=91548]: Done uploading segments... Aug 12 09:18:48.717884761 (87h52m43.148964882s) [upload=91548]: Upload finished. err: <nil>
Duration: 1.868119695s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.282815614s
FAIL (2025-08-12 09:23:57 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 2.282831393s
edge
gateway
us-select-1
|