Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 769544 reports (Page 12319 of 15391)
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [.....X.X......X........X..................X.................]
duration:
5.759719161s
FAIL (2025-08-23 07:34:52 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.759747356s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..........XX................................................]
duration:
5.965555042s
FAIL (2025-08-23 07:34:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.965616289s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
5 out of 60 is failed (threshold=3) [..............X....X.......................X........X...X...]
duration:
1.842344212s
FAIL (2025-08-23 07:34:29 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:32:01.750609660 (350h5m56.181689801s) [upload=364148]: Starting upload
Aug 23 07:32:01.750663393 (350h5m56.181743534s) [upload=364148]: Done starting upload
Aug 23 07:32:01.750718869 (350h5m56.181799s) [upload=364148]: Uploading segments...
Aug 23 07:32:01.750756541 (350h5m56.181836672s) [upload=364148]: Got next segment. Inline: true
Aug 23 07:32:01.750761450 (350h5m56.181841591s) [upload=364148]: Waiting for error group managing segments...
Aug 23 07:32:01.750798622 (350h5m56.181878753s) [upload=364148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:32:01.750803461 (350h5m56.181883592s) [upload=364148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 23 07:32:01.750806396 (350h5m56.181886538s) [upload=364148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 07:32:01.963651980 (350h5m56.394732111s) [upload=364148]: Done uploading segments...
Aug 23 07:32:01.963672149 (350h5m56.39475229s) [upload=364148]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.842364701s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................]
duration:
6.211772964s
FAIL (2025-08-23 07:34:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.211795026s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [......X.X......X........X..................X................]
duration:
5.670148465s
FAIL (2025-08-23 07:34:15 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.670178753s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [...........XX...............................................]
duration:
6.083409057s
FAIL (2025-08-23 07:33:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.083432251s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
5 out of 60 is failed (threshold=3) [...............X....X.......................X........X...X..]
duration:
1.818838306s
FAIL (2025-08-23 07:33:34 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:32:01.750609660 (350h5m56.181689801s) [upload=364148]: Starting upload
Aug 23 07:32:01.750663393 (350h5m56.181743534s) [upload=364148]: Done starting upload
Aug 23 07:32:01.750718869 (350h5m56.181799s) [upload=364148]: Uploading segments...
Aug 23 07:32:01.750756541 (350h5m56.181836672s) [upload=364148]: Got next segment. Inline: true
Aug 23 07:32:01.750761450 (350h5m56.181841591s) [upload=364148]: Waiting for error group managing segments...
Aug 23 07:32:01.750798622 (350h5m56.181878753s) [upload=364148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:32:01.750803461 (350h5m56.181883592s) [upload=364148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 23 07:32:01.750806396 (350h5m56.181886538s) [upload=364148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 07:32:01.963651980 (350h5m56.394732111s) [upload=364148]: Done uploading segments...
Aug 23 07:32:01.963672149 (350h5m56.39475229s) [upload=364148]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.818862581s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................]
duration:
6.716584966s
FAIL (2025-08-23 07:33:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.716610045s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [.......X.X......X........X..................X...............]
duration:
5.701235056s
FAIL (2025-08-23 07:33:14 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.701265254s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................]
duration:
6.364991013s
FAIL (2025-08-23 07:32:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.365009918s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
5 out of 60 is failed (threshold=3) [................X....X.......................X........X...X.]
duration:
1.836006746s
FAIL (2025-08-23 07:32:49 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:32:01.750609660 (350h5m56.181689801s) [upload=364148]: Starting upload
Aug 23 07:32:01.750663393 (350h5m56.181743534s) [upload=364148]: Done starting upload
Aug 23 07:32:01.750718869 (350h5m56.181799s) [upload=364148]: Uploading segments...
Aug 23 07:32:01.750756541 (350h5m56.181836672s) [upload=364148]: Got next segment. Inline: true
Aug 23 07:32:01.750761450 (350h5m56.181841591s) [upload=364148]: Waiting for error group managing segments...
Aug 23 07:32:01.750798622 (350h5m56.181878753s) [upload=364148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:32:01.750803461 (350h5m56.181883592s) [upload=364148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 23 07:32:01.750806396 (350h5m56.181886538s) [upload=364148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 07:32:01.963651980 (350h5m56.394732111s) [upload=364148]: Done uploading segments...
Aug 23 07:32:01.963672149 (350h5m56.39475229s) [upload=364148]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.836028458s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [............XX..............................................]
duration:
5.752450954s
FAIL (2025-08-23 07:32:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.752476023s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [........X.X......X........X..................X..............]
duration:
5.65126423s
FAIL (2025-08-23 07:32:05 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.651290131s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
5 out of 60 is failed (threshold=3) [.................X....X.......................X........X...X]
duration:
860.203663ms
FAIL (2025-08-23 07:32:01 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:32:01.750609660 (350h5m56.181689801s) [upload=364148]: Starting upload
Aug 23 07:32:01.750663393 (350h5m56.181743534s) [upload=364148]: Done starting upload
Aug 23 07:32:01.750718869 (350h5m56.181799s) [upload=364148]: Uploading segments...
Aug 23 07:32:01.750756541 (350h5m56.181836672s) [upload=364148]: Got next segment. Inline: true
Aug 23 07:32:01.750761450 (350h5m56.181841591s) [upload=364148]: Waiting for error group managing segments...
Aug 23 07:32:01.750798622 (350h5m56.181878753s) [upload=364148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:32:01.750803461 (350h5m56.181883592s) [upload=364148]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 23 07:32:01.750806396 (350h5m56.181886538s) [upload=364148]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 07:32:01.963651980 (350h5m56.394732111s) [upload=364148]: Done uploading segments...
Aug 23 07:32:01.963672149 (350h5m56.39475229s) [upload=364148]: Upload finished. err: metaclient: Too Many Requests
Duration: 860.227128ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.............XX.............................................]
duration:
5.851333917s
FAIL (2025-08-23 07:32:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.8513615s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................]
duration:
6.360766397s
FAIL (2025-08-23 07:31:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.360792116s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [.........X.X......X........X..................X.............]
duration:
5.683721266s
FAIL (2025-08-23 07:31:34 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.683750561s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [..................X....X.......................X........X...]
duration:
1.856533501s
FAIL (2025-08-23 07:31:29 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.85658012s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..............XX............................................]
duration:
6.163274943s
FAIL (2025-08-23 07:31:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.163301774s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................]
duration:
6.394218514s
FAIL (2025-08-23 07:30:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.394241079s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [...............XX...........................................]
duration:
5.854144938s
FAIL (2025-08-23 07:30:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.854168353s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [..........X.X......X........X..................X............]
duration:
5.602239939s
FAIL (2025-08-23 07:30:50 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.60226635s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [...................X....X.......................X........X..]
duration:
1.826376215s
FAIL (2025-08-23 07:30:24 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.826473902s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [...........X.X......X........X..................X...........]
duration:
5.607707264s
FAIL (2025-08-23 07:30:08 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.607765086s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [................XX..........................................]
duration:
6.333324757s
FAIL (2025-08-23 07:30:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.333354745s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................]
duration:
6.270384139s
FAIL (2025-08-23 07:29:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.270407723s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.................XX.........................................]
duration:
5.864109303s
FAIL (2025-08-23 07:29:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.864130393s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [....................X....X.......................X........X.]
duration:
1.827511339s
FAIL (2025-08-23 07:29:20 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1.82753694s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [.....................X....X.......................X........X]
duration:
1m0.001250174s
FAIL (2025-08-23 07:29:17 UTC)
orbiter warning: 
operation: StatBucket
======ERROR======
uplink: bucket: metaclient: context deadline exceeded
======LOGS======
Duration: 1m0.001279359s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [............X.X......X........X..................X..........]
duration:
5.720775126s
FAIL (2025-08-23 07:29:05 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.720802288s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................]
duration:
6.795894723s
FAIL (2025-08-23 07:29:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.79591948s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..................XX........................................]
duration:
6.138345265s
FAIL (2025-08-23 07:28:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.138367797s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [.............X.X......X........X..................X.........]
duration:
5.476976819s
FAIL (2025-08-23 07:28:48 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.477011464s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................]
duration:
6.325407603s
FAIL (2025-08-23 07:28:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.325423361s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [...................XX.......................................]
duration:
6.127894038s
FAIL (2025-08-23 07:28:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.127924176s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................]
duration:
6.344075461s
FAIL (2025-08-23 07:27:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.344096651s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [..............X.X......X........X..................X........]
duration:
5.454871118s
FAIL (2025-08-23 07:27:41 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.454900213s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
3 out of 60 is failed (threshold=3) [......................X....X.......................X........]
duration:
2.058871439s
FAIL (2025-08-23 07:27:18 UTC)
orbiter warning: 
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
======LOGS======
Aug 23 07:20:54.505136858 (349h54m48.936216989s) [upload=363957]: Starting upload
Aug 23 07:20:54.505181394 (349h54m48.936261525s) [upload=363957]: Done starting upload
Aug 23 07:20:54.505260215 (349h54m48.936340346s) [upload=363957]: Uploading segments...
Aug 23 07:20:54.505298167 (349h54m48.936378298s) [upload=363957]: Got next segment. Inline: true
Aug 23 07:20:54.505302295 (349h54m48.936382426s) [upload=363957]: Waiting for error group managing segments...
Aug 23 07:20:54.505334997 (349h54m48.936415128s) [upload=363957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:20:54.505338906 (349h54m48.936419037s) [upload=363957]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 23 07:20:54.505341981 (349h54m48.936422112s) [upload=363957]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 07:20:54.815200930 (349h54m49.246281081s) [upload=363957]: Done uploading segments...
Aug 23 07:20:54.815226860 (349h54m49.246307001s) [upload=363957]: Upload finished. err: <nil>
Duration: 2.058894653s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [...............X.X......X........X..................X.......]
duration:
5.814632653s
FAIL (2025-08-23 07:27:17 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.814659765s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [....................XX......................................]
duration:
6.134048212s
FAIL (2025-08-23 07:27:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.134072288s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................]
duration:
6.643922475s
FAIL (2025-08-23 07:27:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.643948766s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [................X.X......X........X..................X......]
duration:
5.525406481s
FAIL (2025-08-23 07:26:43 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.525436057s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [.................X.X......X........X..................X.....]
duration:
5.859190777s
FAIL (2025-08-23 07:26:08 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.859207638s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.....................XX.....................................]
duration:
6.180128665s
FAIL (2025-08-23 07:26:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.180152078s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................]
duration:
6.623083684s
FAIL (2025-08-23 07:26:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.62312291s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
3 out of 60 is failed (threshold=3) [.......................X....X.......................X.......]
duration:
1.859101444s
FAIL (2025-08-23 07:26:04 UTC)
orbiter warning: 
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
======LOGS======
Aug 23 07:20:54.505136858 (349h54m48.936216989s) [upload=363957]: Starting upload
Aug 23 07:20:54.505181394 (349h54m48.936261525s) [upload=363957]: Done starting upload
Aug 23 07:20:54.505260215 (349h54m48.936340346s) [upload=363957]: Uploading segments...
Aug 23 07:20:54.505298167 (349h54m48.936378298s) [upload=363957]: Got next segment. Inline: true
Aug 23 07:20:54.505302295 (349h54m48.936382426s) [upload=363957]: Waiting for error group managing segments...
Aug 23 07:20:54.505334997 (349h54m48.936415128s) [upload=363957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:20:54.505338906 (349h54m48.936419037s) [upload=363957]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 23 07:20:54.505341981 (349h54m48.936422112s) [upload=363957]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 07:20:54.815200930 (349h54m49.246281081s) [upload=363957]: Done uploading segments...
Aug 23 07:20:54.815226860 (349h54m49.246307001s) [upload=363957]: Upload finished. err: <nil>
Duration: 1.859152441s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [......................XX....................................]
duration:
5.828022712s
FAIL (2025-08-23 07:25:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.828045345s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
3 out of 60 is failed (threshold=3) [........................X....X.......................X......]
duration:
1.889448981s
FAIL (2025-08-23 07:25:08 UTC)
orbiter warning: 
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K
======LOGS======
Aug 23 07:20:54.505136858 (349h54m48.936216989s) [upload=363957]: Starting upload
Aug 23 07:20:54.505181394 (349h54m48.936261525s) [upload=363957]: Done starting upload
Aug 23 07:20:54.505260215 (349h54m48.936340346s) [upload=363957]: Uploading segments...
Aug 23 07:20:54.505298167 (349h54m48.936378298s) [upload=363957]: Got next segment. Inline: true
Aug 23 07:20:54.505302295 (349h54m48.936382426s) [upload=363957]: Waiting for error group managing segments...
Aug 23 07:20:54.505334997 (349h54m48.936415128s) [upload=363957]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:20:54.505338906 (349h54m48.936419037s) [upload=363957]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 23 07:20:54.505341981 (349h54m48.936422112s) [upload=363957]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 23 07:20:54.815200930 (349h54m49.246281081s) [upload=363957]: Done uploading segments...
Aug 23 07:20:54.815226860 (349h54m49.246307001s) [upload=363957]: Upload finished. err: <nil>
Duration: 1.889473909s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................]
duration:
6.333624268s
FAIL (2025-08-23 07:25:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.333649225s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
5 out of 60 is failed (threshold=3) [..................X.X......X........X..................X....]
duration:
5.66460839s
FAIL (2025-08-23 07:25:00 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 23 07:21:57.296219160 (349h55m51.727299291s) [upload=363978]: Starting upload
Aug 23 07:21:57.296310955 (349h55m51.727391086s) [upload=363978]: Done starting upload
Aug 23 07:21:57.296422659 (349h55m51.72750279s) [upload=363978]: Uploading segments...
Aug 23 07:21:57.296518613 (349h55m51.727598744s) [upload=363978]: Got next segment. Inline: false
Aug 23 07:21:57.296566725 (349h55m51.727646856s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 23 07:21:57.296571554 (349h55m51.727651685s) [upload=363978]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 23 07:21:57.569135802 (349h55m52.000215933s) [upload=363978]: Next segment err: metaclient: Too Many Requests
Aug 23 07:21:57.569167323 (349h55m52.000247454s) [upload=363978]: Done uploading segments...
Aug 23 07:21:57.569184676 (349h55m52.000264807s) [upload=363978]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.664630323s core uplink us-select-1 us1-placement-20 metainfo-projects