Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
6.323093767s
FAIL (2025-08-19 11:17:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.323113324s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [............X......................................X..X.....] duration:
6.015741322s
FAIL (2025-08-19 11:17:03 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 11:13:02.350167148 (257h46m56.781247279s) [upload=268051]: Starting upload Aug 19 11:13:02.350220850 (257h46m56.781300981s) [upload=268051]: Done starting upload Aug 19 11:13:02.350300823 (257h46m56.781380964s) [upload=268051]: Uploading segments... Aug 19 11:13:02.350427946 (257h46m56.781508087s) [upload=268051]: Got next segment. Inline: false Aug 19 11:13:02.350462212 (257h46m56.781542343s) [upload=268051]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 11:13:02.350467132 (257h46m56.781547263s) [upload=268051]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 19 11:13:02.564249157 (257h46m56.995329288s) [upload=268051]: Next segment err: metaclient: Too Many Requests Aug 19 11:13:02.564267602 (257h46m56.995347733s) [upload=268051]: Done uploading segments... Aug 19 11:13:02.564303270 (257h46m56.995383401s) [upload=268051]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.015765459s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.....X..X.......X........................X..................] duration:
1.898278497s
FAIL (2025-08-19 11:16:42 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 1.898312081s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.............X......................................X..X....] duration:
5.91701641s
FAIL (2025-08-19 11:16:34 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 11:13:02.350167148 (257h46m56.781247279s) [upload=268051]: Starting upload Aug 19 11:13:02.350220850 (257h46m56.781300981s) [upload=268051]: Done starting upload Aug 19 11:13:02.350300823 (257h46m56.781380964s) [upload=268051]: Uploading segments... Aug 19 11:13:02.350427946 (257h46m56.781508087s) [upload=268051]: Got next segment. Inline: false Aug 19 11:13:02.350462212 (257h46m56.781542343s) [upload=268051]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 11:13:02.350467132 (257h46m56.781547263s) [upload=268051]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 19 11:13:02.564249157 (257h46m56.995329288s) [upload=268051]: Next segment err: metaclient: Too Many Requests Aug 19 11:13:02.564267602 (257h46m56.995347733s) [upload=268051]: Done uploading segments... Aug 19 11:13:02.564303270 (257h46m56.995383401s) [upload=268051]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.917113275s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
6.465949783s
FAIL (2025-08-19 11:16:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.465979851s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [..............X......................................X..X...] duration:
5.637218709s
FAIL (2025-08-19 11:16:15 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 11:13:02.350167148 (257h46m56.781247279s) [upload=268051]: Starting upload Aug 19 11:13:02.350220850 (257h46m56.781300981s) [upload=268051]: Done starting upload Aug 19 11:13:02.350300823 (257h46m56.781380964s) [upload=268051]: Uploading segments... Aug 19 11:13:02.350427946 (257h46m56.781508087s) [upload=268051]: Got next segment. Inline: false Aug 19 11:13:02.350462212 (257h46m56.781542343s) [upload=268051]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 11:13:02.350467132 (257h46m56.781547263s) [upload=268051]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 19 11:13:02.564249157 (257h46m56.995329288s) [upload=268051]: Next segment err: metaclient: Too Many Requests Aug 19 11:13:02.564267602 (257h46m56.995347733s) [upload=268051]: Done uploading segments... Aug 19 11:13:02.564303270 (257h46m56.995383401s) [upload=268051]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.637241532s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [......X..X.......X........................X.................] duration:
2.041704915s
FAIL (2025-08-19 11:15:49 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 2.0417389s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.......X..X.......X........................X................] duration:
1.91739256s
FAIL (2025-08-19 11:15:29 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 1.917435913s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [...............X......................................X..X..] duration:
5.697227593s
FAIL (2025-08-19 11:15:18 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 11:13:02.350167148 (257h46m56.781247279s) [upload=268051]: Starting upload Aug 19 11:13:02.350220850 (257h46m56.781300981s) [upload=268051]: Done starting upload Aug 19 11:13:02.350300823 (257h46m56.781380964s) [upload=268051]: Uploading segments... Aug 19 11:13:02.350427946 (257h46m56.781508087s) [upload=268051]: Got next segment. Inline: false Aug 19 11:13:02.350462212 (257h46m56.781542343s) [upload=268051]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 11:13:02.350467132 (257h46m56.781547263s) [upload=268051]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 19 11:13:02.564249157 (257h46m56.995329288s) [upload=268051]: Next segment err: metaclient: Too Many Requests Aug 19 11:13:02.564267602 (257h46m56.995347733s) [upload=268051]: Done uploading segments... Aug 19 11:13:02.564303270 (257h46m56.995383401s) [upload=268051]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.697253062s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
6.522261088s
FAIL (2025-08-19 11:15:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.522287488s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
6.434479545s
FAIL (2025-08-19 11:14:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.434500525s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [........X..X.......X........................X...............] duration:
2.045263263s
FAIL (2025-08-19 11:14:22 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 2.045295205s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [................X......................................X..X.] duration:
6.518645754s
FAIL (2025-08-19 11:14:06 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 11:13:02.350167148 (257h46m56.781247279s) [upload=268051]: Starting upload Aug 19 11:13:02.350220850 (257h46m56.781300981s) [upload=268051]: Done starting upload Aug 19 11:13:02.350300823 (257h46m56.781380964s) [upload=268051]: Uploading segments... Aug 19 11:13:02.350427946 (257h46m56.781508087s) [upload=268051]: Got next segment. Inline: false Aug 19 11:13:02.350462212 (257h46m56.781542343s) [upload=268051]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 11:13:02.350467132 (257h46m56.781547263s) [upload=268051]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 19 11:13:02.564249157 (257h46m56.995329288s) [upload=268051]: Next segment err: metaclient: Too Many Requests Aug 19 11:13:02.564267602 (257h46m56.995347733s) [upload=268051]: Done uploading segments... Aug 19 11:13:02.564303270 (257h46m56.995383401s) [upload=268051]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.518670933s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.........X..X.......X........................X..............] duration:
1.965108755s
FAIL (2025-08-19 11:13:50 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 1.965134846s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
6.528959675s
FAIL (2025-08-19 11:13:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.528982067s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..........X..X.......X........................X.............] duration:
1.822412725s
FAIL (2025-08-19 11:13:23 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 1.822449887s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [.................X......................................X..X] duration:
862.148926ms
FAIL (2025-08-19 11:13:02 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 11:13:02.350167148 (257h46m56.781247279s) [upload=268051]: Starting upload Aug 19 11:13:02.350220850 (257h46m56.781300981s) [upload=268051]: Done starting upload Aug 19 11:13:02.350300823 (257h46m56.781380964s) [upload=268051]: Uploading segments... Aug 19 11:13:02.350427946 (257h46m56.781508087s) [upload=268051]: Got next segment. Inline: false Aug 19 11:13:02.350462212 (257h46m56.781542343s) [upload=268051]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 19 11:13:02.350467132 (257h46m56.781547263s) [upload=268051]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 19 11:13:02.564249157 (257h46m56.995329288s) [upload=268051]: Next segment err: metaclient: Too Many Requests Aug 19 11:13:02.564267602 (257h46m56.995347733s) [upload=268051]: Done uploading segments... Aug 19 11:13:02.564303270 (257h46m56.995383401s) [upload=268051]: Upload finished. err: metaclient: Too Many Requests
Duration: 862.162441ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................X......................................X..] duration:
5.773892924s
FAIL (2025-08-19 11:12:51 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 11:10:31.268789228 (257h44m25.699869359s) [upload=268006]: Starting upload
Aug 19 11:10:31.268832121 (257h44m25.699912242s) [upload=268006]: Done starting upload
Aug 19 11:10:31.268997126 (257h44m25.700077257s) [upload=268006]: Uploading segments...
Aug 19 11:10:31.269032193 (257h44m25.700112324s) [upload=268006]: Got next segment. Inline: false
Aug 19 11:10:31.269288364 (257h44m25.700368495s) [upload=268006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:10:31.269294115 (257h44m25.700374246s) [upload=268006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 11:10:31.774988531 (257h44m26.206068662s) [seg_pos={0 0}, upload=268006]: Begin upload segment...
Aug 19 11:10:31.775038868 (257h44m26.206118999s) [seg_pos={0 0}, upload=268006]: Done begin upload segment.
Aug 19 11:10:31.775061500 (257h44m26.206141631s) [noise=true , address=45.91.116.6:4996 , piece_id=T4ZQG3LPULEXWBT2FUG4G5PSF5VB5HTIWN7F6JR6CMS3Q3XFQ6AA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.775098792 (257h44m26.206178923s) [noise=true , address=77.90.31.18:4808 , piece_id=4F6AMQMVBQXCQFLTXM5PCSUZZLH44DX5CQJ75CCLFQ573KBT5E5A, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.775423233 (257h44m26.206503364s) [noise=true , address=104.36.82.23:4470 , piece_id=AICIAJGD4NUYTNHNNZU3RY3QC6QZUUVBG33CX44URVJNRKSZEKZQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.775528524 (257h44m26.206608655s) [noise=true , address=77.90.31.17:4799 , piece_id=6R3TF3GHRKXNMCIALZNVLQVCRYCRJGNXYHEDWAXWONMLTNJDN3MA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.775748826 (257h44m26.206828957s) [noise=true , address=45.91.116.12:4229 , piece_id=BCGJP7MCGW4ASHTAN7IGDZZA67IYV47R7IKTCXLHZJJEZNZPU7BQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.776047347 (257h44m26.207127478s) [noise=true , address=77.90.31.21:4903 , piece_id=OW2SO4EZYVXB7R6AWRZS2AZLBCN3TT7NOLIJATFQ6F6SNVGAY7IQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.776296965 (257h44m26.207377096s) [noise=true , address=104.36.82.6:4908 , piece_id=YIPIUCCT5CILBJYX4QFL7SJXEEMUI4K56KZGW32PNO3UA75TKVJA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.776598292 (257h44m26.207678423s) [noise=true , address=45.91.116.14:4220 , piece_id=E6UPNYACCED4KHG6SI57N5FWUQ7NWPUD6YTHY63BEU7LOJ4HBKVA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.776859200 (257h44m26.207939322s) [noise=true , address=77.90.31.19:4876 , piece_id=KXYZROJUTF4C5CTPRLKWL6P4WLSIFP4BKBTOPUBOAVHTABMXKQ7Q, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.777090453 (257h44m26.208170584s) [noise=true , address=79.127.219.38:4027 , piece_id=CR4BD5LEHVB3DL2QO4G2HJ6ERIMKAPIYLQ5S5Z7L3ZYXI7K6U2XQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.777500116 (257h44m26.208580248s) [noise=true , address=45.91.116.5:4301 , piece_id=VGGQTH6ITD5SEFRDLDTU6APZI2APNPZ3Q5DJJRC6P2BTX237VTEA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.778120113 (257h44m26.209200244s) [noise=true , address=77.90.31.13:4644 , piece_id=NOUOHAODVLQDH6AQHMAP7UCQ54Q235PODYSH6C254VDJGPKINGXQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.778636361 (257h44m26.209716492s) [noise=true , address=77.90.31.10:4538 , piece_id=W65SVEGF4OTV24Z6MGOPMXCE34WBLUNEWRRDADFPOQXBZVXGSODQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.779150395 (257h44m26.210230526s) [noise=true , address=77.90.31.15:4709 , piece_id=4URFK2HRW43ED66DVTDQIXQOB7YFBTFVV5D6KFSDUL23KJFOJCYQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.779459807 (257h44m26.210539938s) [noise=true , address=45.91.116.13:4161 , piece_id=7X2RJXTNUGBAFHTDWKMXTZM3BPHCFGX3KPDOVTG4GRCKQ5C6GPPQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.779701579 (257h44m26.21078171s) [noise=true , address=104.36.82.60:4361 , piece_id=3XV3AZONK3457X5RDQG254NCUW4HE6HO76VFNKT5XSFPXX4FE2BQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.779968750 (257h44m26.211048881s) [noise=true , address=45.91.116.9:4267 , piece_id=TTOGGN6SOWJGZO4RK5RRIROMOPAEJP77I3PD62PT3YE54R5Y4HLA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.780226864 (257h44m26.211306995s) [noise=true , address=79.127.219.51:4215 , piece_id=VIF6J4NJVTL5PLLBK4BJEZN36CNFMLYS65F6JAIZFHA52CXOJCTQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:33.270967621 (257h44m27.702047742s) [upload=268006]: Next returned nil segment
Aug 19 11:10:33.270983160 (257h44m27.702063291s) [upload=268006]: Waiting for error group managing segments...
Aug 19 11:10:33.659531166 (257h44m28.090611297s) [noise=true , address=79.127.219.38:4027 , piece_id=CR4BD5LEHVB3DL2QO4G2HJ6ERIMKAPIYLQ5S5Z7L3ZYXI7K6U2XQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.701294538 (257h44m28.132374669s) [noise=true , address=104.36.82.60:4361 , piece_id=3XV3AZONK3457X5RDQG254NCUW4HE6HO76VFNKT5XSFPXX4FE2BQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.705700798 (257h44m28.136780919s) [noise=true , address=79.127.219.51:4215 , piece_id=VIF6J4NJVTL5PLLBK4BJEZN36CNFMLYS65F6JAIZFHA52CXOJCTQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.708143333 (257h44m28.139223464s) [noise=true , address=77.90.31.18:4808 , piece_id=4F6AMQMVBQXCQFLTXM5PCSUZZLH44DX5CQJ75CCLFQ573KBT5E5A, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.731526073 (257h44m28.162606194s) [noise=true , address=77.90.31.15:4709 , piece_id=4URFK2HRW43ED66DVTDQIXQOB7YFBTFVV5D6KFSDUL23KJFOJCYQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.762966121 (257h44m28.194046242s) [noise=true , address=77.90.31.17:4799 , piece_id=6R3TF3GHRKXNMCIALZNVLQVCRYCRJGNXYHEDWAXWONMLTNJDN3MA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.783901640 (257h44m28.214981761s) [noise=true , address=104.36.82.6:4908 , piece_id=YIPIUCCT5CILBJYX4QFL7SJXEEMUI4K56KZGW32PNO3UA75TKVJA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.790729604 (257h44m28.221809736s) [noise=true , address=45.91.116.6:4996 , piece_id=T4ZQG3LPULEXWBT2FUG4G5PSF5VB5HTIWN7F6JR6CMS3Q3XFQ6AA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.813267900 (257h44m28.244348021s) [noise=true , address=104.36.82.23:4470 , piece_id=AICIAJGD4NUYTNHNNZU3RY3QC6QZUUVBG33CX44URVJNRKSZEKZQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.815730592 (257h44m28.246810723s) [noise=true , address=45.91.116.5:4301 , piece_id=VGGQTH6ITD5SEFRDLDTU6APZI2APNPZ3Q5DJJRC6P2BTX237VTEA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.830686509 (257h44m28.26176664s) [noise=true , address=77.90.31.10:4538 , piece_id=W65SVEGF4OTV24Z6MGOPMXCE34WBLUNEWRRDADFPOQXBZVXGSODQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.840174453 (257h44m28.271254584s) [noise=true , address=45.91.116.9:4267 , piece_id=TTOGGN6SOWJGZO4RK5RRIROMOPAEJP77I3PD62PT3YE54R5Y4HLA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.840496980 (257h44m28.271577111s) [noise=true , address=45.91.116.14:4220 , piece_id=E6UPNYACCED4KHG6SI57N5FWUQ7NWPUD6YTHY63BEU7LOJ4HBKVA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.867275459 (257h44m28.29835558s) [noise=true , address=45.91.116.12:4229 , piece_id=BCGJP7MCGW4ASHTAN7IGDZZA67IYV47R7IKTCXLHZJJEZNZPU7BQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.867302751 (257h44m28.298382882s) [seg_pos={0 0}, upload=268006]: Segment reached optimal threshold of 14 pieces.
Aug 19 11:10:33.867471283 (257h44m28.298551404s) [noise=true , address=77.90.31.21:4903 , piece_id=OW2SO4EZYVXB7R6AWRZS2AZLBCN3TT7NOLIJATFQ6F6SNVGAY7IQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZYWQ1a2sTKnFpF28iQVrem2s8nSpXkeJzZDaS7jvTZNANdfp): protocol: expected piece hash; context canceled; EOF
Aug 19 11:10:33.867516500 (257h44m28.298596631s) [noise=true , address=77.90.31.19:4876 , piece_id=KXYZROJUTF4C5CTPRLKWL6P4WLSIFP4BKBTOPUBOAVHTABMXKQ7Q, seg_pos={0 0}, upload=268006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 11:10:33.867560314 (257h44m28.298640445s) [noise=true , address=45.91.116.13:4161 , piece_id=7X2RJXTNUGBAFHTDWKMXTZM3BPHCFGX3KPDOVTG4GRCKQ5C6GPPQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TaXapd6q5Vh1CQ5tQP98iQqVY99HFdGGv8CebGq49D9aKR9n2): protocol: expected piece hash; context canceled; EOF
Aug 19 11:10:33.867646239 (257h44m28.29872637s) [noise=true , address=77.90.31.13:4644 , piece_id=NOUOHAODVLQDH6AQHMAP7UCQ54Q235PODYSH6C254VDJGPKINGXQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13QCH91kRMQjVj2ASsVDcyXQ1aapQRVeUm6MMCGimsR6Ns973h): protocol: write tcp 188.166.225.170:32990->77.90.31.13:4644: use of closed network connection; write tcp 188.166.225.170:32990->77.90.31.13:4644: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32990->77.90.31.13:4644: use of closed network connection
Aug 19 11:10:33.867666487 (257h44m28.298746618s) [seg_pos={0 0}, upload=268006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 11:10:33.867739918 (257h44m28.298820049s) [upload=268006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 11:10:33.867744586 (257h44m28.298824717s) [upload=268006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:10:34.562058994 (257h44m28.993139125s) [upload=268006]: Done uploading segments...
Aug 19 11:10:34.562077750 (257h44m28.993157881s) [upload=268006]: Upload finished. err: <nil>
Duration: 5.773914606s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
7.53572379s
FAIL (2025-08-19 11:12:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.535744217s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [...........X..X.......X........................X............] duration:
1.957244937s
FAIL (2025-08-19 11:12:31 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 1.957277338s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................X......................................X.] duration:
5.903223277s
FAIL (2025-08-19 11:11:51 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 11:10:31.268789228 (257h44m25.699869359s) [upload=268006]: Starting upload
Aug 19 11:10:31.268832121 (257h44m25.699912242s) [upload=268006]: Done starting upload
Aug 19 11:10:31.268997126 (257h44m25.700077257s) [upload=268006]: Uploading segments...
Aug 19 11:10:31.269032193 (257h44m25.700112324s) [upload=268006]: Got next segment. Inline: false
Aug 19 11:10:31.269288364 (257h44m25.700368495s) [upload=268006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:10:31.269294115 (257h44m25.700374246s) [upload=268006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 11:10:31.774988531 (257h44m26.206068662s) [seg_pos={0 0}, upload=268006]: Begin upload segment...
Aug 19 11:10:31.775038868 (257h44m26.206118999s) [seg_pos={0 0}, upload=268006]: Done begin upload segment.
Aug 19 11:10:31.775061500 (257h44m26.206141631s) [noise=true , address=45.91.116.6:4996 , piece_id=T4ZQG3LPULEXWBT2FUG4G5PSF5VB5HTIWN7F6JR6CMS3Q3XFQ6AA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.775098792 (257h44m26.206178923s) [noise=true , address=77.90.31.18:4808 , piece_id=4F6AMQMVBQXCQFLTXM5PCSUZZLH44DX5CQJ75CCLFQ573KBT5E5A, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.775423233 (257h44m26.206503364s) [noise=true , address=104.36.82.23:4470 , piece_id=AICIAJGD4NUYTNHNNZU3RY3QC6QZUUVBG33CX44URVJNRKSZEKZQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.775528524 (257h44m26.206608655s) [noise=true , address=77.90.31.17:4799 , piece_id=6R3TF3GHRKXNMCIALZNVLQVCRYCRJGNXYHEDWAXWONMLTNJDN3MA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.775748826 (257h44m26.206828957s) [noise=true , address=45.91.116.12:4229 , piece_id=BCGJP7MCGW4ASHTAN7IGDZZA67IYV47R7IKTCXLHZJJEZNZPU7BQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.776047347 (257h44m26.207127478s) [noise=true , address=77.90.31.21:4903 , piece_id=OW2SO4EZYVXB7R6AWRZS2AZLBCN3TT7NOLIJATFQ6F6SNVGAY7IQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.776296965 (257h44m26.207377096s) [noise=true , address=104.36.82.6:4908 , piece_id=YIPIUCCT5CILBJYX4QFL7SJXEEMUI4K56KZGW32PNO3UA75TKVJA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.776598292 (257h44m26.207678423s) [noise=true , address=45.91.116.14:4220 , piece_id=E6UPNYACCED4KHG6SI57N5FWUQ7NWPUD6YTHY63BEU7LOJ4HBKVA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.776859200 (257h44m26.207939322s) [noise=true , address=77.90.31.19:4876 , piece_id=KXYZROJUTF4C5CTPRLKWL6P4WLSIFP4BKBTOPUBOAVHTABMXKQ7Q, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.777090453 (257h44m26.208170584s) [noise=true , address=79.127.219.38:4027 , piece_id=CR4BD5LEHVB3DL2QO4G2HJ6ERIMKAPIYLQ5S5Z7L3ZYXI7K6U2XQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.777500116 (257h44m26.208580248s) [noise=true , address=45.91.116.5:4301 , piece_id=VGGQTH6ITD5SEFRDLDTU6APZI2APNPZ3Q5DJJRC6P2BTX237VTEA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.778120113 (257h44m26.209200244s) [noise=true , address=77.90.31.13:4644 , piece_id=NOUOHAODVLQDH6AQHMAP7UCQ54Q235PODYSH6C254VDJGPKINGXQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.778636361 (257h44m26.209716492s) [noise=true , address=77.90.31.10:4538 , piece_id=W65SVEGF4OTV24Z6MGOPMXCE34WBLUNEWRRDADFPOQXBZVXGSODQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.779150395 (257h44m26.210230526s) [noise=true , address=77.90.31.15:4709 , piece_id=4URFK2HRW43ED66DVTDQIXQOB7YFBTFVV5D6KFSDUL23KJFOJCYQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.779459807 (257h44m26.210539938s) [noise=true , address=45.91.116.13:4161 , piece_id=7X2RJXTNUGBAFHTDWKMXTZM3BPHCFGX3KPDOVTG4GRCKQ5C6GPPQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.779701579 (257h44m26.21078171s) [noise=true , address=104.36.82.60:4361 , piece_id=3XV3AZONK3457X5RDQG254NCUW4HE6HO76VFNKT5XSFPXX4FE2BQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.779968750 (257h44m26.211048881s) [noise=true , address=45.91.116.9:4267 , piece_id=TTOGGN6SOWJGZO4RK5RRIROMOPAEJP77I3PD62PT3YE54R5Y4HLA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.780226864 (257h44m26.211306995s) [noise=true , address=79.127.219.51:4215 , piece_id=VIF6J4NJVTL5PLLBK4BJEZN36CNFMLYS65F6JAIZFHA52CXOJCTQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:33.270967621 (257h44m27.702047742s) [upload=268006]: Next returned nil segment
Aug 19 11:10:33.270983160 (257h44m27.702063291s) [upload=268006]: Waiting for error group managing segments...
Aug 19 11:10:33.659531166 (257h44m28.090611297s) [noise=true , address=79.127.219.38:4027 , piece_id=CR4BD5LEHVB3DL2QO4G2HJ6ERIMKAPIYLQ5S5Z7L3ZYXI7K6U2XQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.701294538 (257h44m28.132374669s) [noise=true , address=104.36.82.60:4361 , piece_id=3XV3AZONK3457X5RDQG254NCUW4HE6HO76VFNKT5XSFPXX4FE2BQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.705700798 (257h44m28.136780919s) [noise=true , address=79.127.219.51:4215 , piece_id=VIF6J4NJVTL5PLLBK4BJEZN36CNFMLYS65F6JAIZFHA52CXOJCTQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.708143333 (257h44m28.139223464s) [noise=true , address=77.90.31.18:4808 , piece_id=4F6AMQMVBQXCQFLTXM5PCSUZZLH44DX5CQJ75CCLFQ573KBT5E5A, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.731526073 (257h44m28.162606194s) [noise=true , address=77.90.31.15:4709 , piece_id=4URFK2HRW43ED66DVTDQIXQOB7YFBTFVV5D6KFSDUL23KJFOJCYQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.762966121 (257h44m28.194046242s) [noise=true , address=77.90.31.17:4799 , piece_id=6R3TF3GHRKXNMCIALZNVLQVCRYCRJGNXYHEDWAXWONMLTNJDN3MA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.783901640 (257h44m28.214981761s) [noise=true , address=104.36.82.6:4908 , piece_id=YIPIUCCT5CILBJYX4QFL7SJXEEMUI4K56KZGW32PNO3UA75TKVJA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.790729604 (257h44m28.221809736s) [noise=true , address=45.91.116.6:4996 , piece_id=T4ZQG3LPULEXWBT2FUG4G5PSF5VB5HTIWN7F6JR6CMS3Q3XFQ6AA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.813267900 (257h44m28.244348021s) [noise=true , address=104.36.82.23:4470 , piece_id=AICIAJGD4NUYTNHNNZU3RY3QC6QZUUVBG33CX44URVJNRKSZEKZQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.815730592 (257h44m28.246810723s) [noise=true , address=45.91.116.5:4301 , piece_id=VGGQTH6ITD5SEFRDLDTU6APZI2APNPZ3Q5DJJRC6P2BTX237VTEA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.830686509 (257h44m28.26176664s) [noise=true , address=77.90.31.10:4538 , piece_id=W65SVEGF4OTV24Z6MGOPMXCE34WBLUNEWRRDADFPOQXBZVXGSODQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.840174453 (257h44m28.271254584s) [noise=true , address=45.91.116.9:4267 , piece_id=TTOGGN6SOWJGZO4RK5RRIROMOPAEJP77I3PD62PT3YE54R5Y4HLA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.840496980 (257h44m28.271577111s) [noise=true , address=45.91.116.14:4220 , piece_id=E6UPNYACCED4KHG6SI57N5FWUQ7NWPUD6YTHY63BEU7LOJ4HBKVA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.867275459 (257h44m28.29835558s) [noise=true , address=45.91.116.12:4229 , piece_id=BCGJP7MCGW4ASHTAN7IGDZZA67IYV47R7IKTCXLHZJJEZNZPU7BQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.867302751 (257h44m28.298382882s) [seg_pos={0 0}, upload=268006]: Segment reached optimal threshold of 14 pieces.
Aug 19 11:10:33.867471283 (257h44m28.298551404s) [noise=true , address=77.90.31.21:4903 , piece_id=OW2SO4EZYVXB7R6AWRZS2AZLBCN3TT7NOLIJATFQ6F6SNVGAY7IQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZYWQ1a2sTKnFpF28iQVrem2s8nSpXkeJzZDaS7jvTZNANdfp): protocol: expected piece hash; context canceled; EOF
Aug 19 11:10:33.867516500 (257h44m28.298596631s) [noise=true , address=77.90.31.19:4876 , piece_id=KXYZROJUTF4C5CTPRLKWL6P4WLSIFP4BKBTOPUBOAVHTABMXKQ7Q, seg_pos={0 0}, upload=268006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 11:10:33.867560314 (257h44m28.298640445s) [noise=true , address=45.91.116.13:4161 , piece_id=7X2RJXTNUGBAFHTDWKMXTZM3BPHCFGX3KPDOVTG4GRCKQ5C6GPPQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TaXapd6q5Vh1CQ5tQP98iQqVY99HFdGGv8CebGq49D9aKR9n2): protocol: expected piece hash; context canceled; EOF
Aug 19 11:10:33.867646239 (257h44m28.29872637s) [noise=true , address=77.90.31.13:4644 , piece_id=NOUOHAODVLQDH6AQHMAP7UCQ54Q235PODYSH6C254VDJGPKINGXQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13QCH91kRMQjVj2ASsVDcyXQ1aapQRVeUm6MMCGimsR6Ns973h): protocol: write tcp 188.166.225.170:32990->77.90.31.13:4644: use of closed network connection; write tcp 188.166.225.170:32990->77.90.31.13:4644: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32990->77.90.31.13:4644: use of closed network connection
Aug 19 11:10:33.867666487 (257h44m28.298746618s) [seg_pos={0 0}, upload=268006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 11:10:33.867739918 (257h44m28.298820049s) [upload=268006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 11:10:33.867744586 (257h44m28.298824717s) [upload=268006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:10:34.562058994 (257h44m28.993139125s) [upload=268006]: Done uploading segments...
Aug 19 11:10:34.562077750 (257h44m28.993157881s) [upload=268006]: Upload finished. err: <nil>
Duration: 5.903247432s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [............X..X.......X........................X...........] duration:
2.087171469s
FAIL (2025-08-19 11:11:51 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 2.087197658s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
6.508005435s
FAIL (2025-08-19 11:11:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.508025452s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.............X..X.......X........................X..........] duration:
1.916724443s
FAIL (2025-08-19 11:11:33 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 1.916749461s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..............X..X.......X........................X.........] duration:
1.837136888s
FAIL (2025-08-19 11:10:53 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 1.837159732s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
6.819317104s
FAIL (2025-08-19 11:10:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.819337765s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................X......................................X] duration:
4.460137667s
FAIL (2025-08-19 11:10:35 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 11:10:31.268789228 (257h44m25.699869359s) [upload=268006]: Starting upload
Aug 19 11:10:31.268832121 (257h44m25.699912242s) [upload=268006]: Done starting upload
Aug 19 11:10:31.268997126 (257h44m25.700077257s) [upload=268006]: Uploading segments...
Aug 19 11:10:31.269032193 (257h44m25.700112324s) [upload=268006]: Got next segment. Inline: false
Aug 19 11:10:31.269288364 (257h44m25.700368495s) [upload=268006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:10:31.269294115 (257h44m25.700374246s) [upload=268006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 11:10:31.774988531 (257h44m26.206068662s) [seg_pos={0 0}, upload=268006]: Begin upload segment...
Aug 19 11:10:31.775038868 (257h44m26.206118999s) [seg_pos={0 0}, upload=268006]: Done begin upload segment.
Aug 19 11:10:31.775061500 (257h44m26.206141631s) [noise=true , address=45.91.116.6:4996 , piece_id=T4ZQG3LPULEXWBT2FUG4G5PSF5VB5HTIWN7F6JR6CMS3Q3XFQ6AA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.775098792 (257h44m26.206178923s) [noise=true , address=77.90.31.18:4808 , piece_id=4F6AMQMVBQXCQFLTXM5PCSUZZLH44DX5CQJ75CCLFQ573KBT5E5A, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.775423233 (257h44m26.206503364s) [noise=true , address=104.36.82.23:4470 , piece_id=AICIAJGD4NUYTNHNNZU3RY3QC6QZUUVBG33CX44URVJNRKSZEKZQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.775528524 (257h44m26.206608655s) [noise=true , address=77.90.31.17:4799 , piece_id=6R3TF3GHRKXNMCIALZNVLQVCRYCRJGNXYHEDWAXWONMLTNJDN3MA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.775748826 (257h44m26.206828957s) [noise=true , address=45.91.116.12:4229 , piece_id=BCGJP7MCGW4ASHTAN7IGDZZA67IYV47R7IKTCXLHZJJEZNZPU7BQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.776047347 (257h44m26.207127478s) [noise=true , address=77.90.31.21:4903 , piece_id=OW2SO4EZYVXB7R6AWRZS2AZLBCN3TT7NOLIJATFQ6F6SNVGAY7IQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.776296965 (257h44m26.207377096s) [noise=true , address=104.36.82.6:4908 , piece_id=YIPIUCCT5CILBJYX4QFL7SJXEEMUI4K56KZGW32PNO3UA75TKVJA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.776598292 (257h44m26.207678423s) [noise=true , address=45.91.116.14:4220 , piece_id=E6UPNYACCED4KHG6SI57N5FWUQ7NWPUD6YTHY63BEU7LOJ4HBKVA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.776859200 (257h44m26.207939322s) [noise=true , address=77.90.31.19:4876 , piece_id=KXYZROJUTF4C5CTPRLKWL6P4WLSIFP4BKBTOPUBOAVHTABMXKQ7Q, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.777090453 (257h44m26.208170584s) [noise=true , address=79.127.219.38:4027 , piece_id=CR4BD5LEHVB3DL2QO4G2HJ6ERIMKAPIYLQ5S5Z7L3ZYXI7K6U2XQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.777500116 (257h44m26.208580248s) [noise=true , address=45.91.116.5:4301 , piece_id=VGGQTH6ITD5SEFRDLDTU6APZI2APNPZ3Q5DJJRC6P2BTX237VTEA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.778120113 (257h44m26.209200244s) [noise=true , address=77.90.31.13:4644 , piece_id=NOUOHAODVLQDH6AQHMAP7UCQ54Q235PODYSH6C254VDJGPKINGXQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.778636361 (257h44m26.209716492s) [noise=true , address=77.90.31.10:4538 , piece_id=W65SVEGF4OTV24Z6MGOPMXCE34WBLUNEWRRDADFPOQXBZVXGSODQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.779150395 (257h44m26.210230526s) [noise=true , address=77.90.31.15:4709 , piece_id=4URFK2HRW43ED66DVTDQIXQOB7YFBTFVV5D6KFSDUL23KJFOJCYQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.779459807 (257h44m26.210539938s) [noise=true , address=45.91.116.13:4161 , piece_id=7X2RJXTNUGBAFHTDWKMXTZM3BPHCFGX3KPDOVTG4GRCKQ5C6GPPQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.779701579 (257h44m26.21078171s) [noise=true , address=104.36.82.60:4361 , piece_id=3XV3AZONK3457X5RDQG254NCUW4HE6HO76VFNKT5XSFPXX4FE2BQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.779968750 (257h44m26.211048881s) [noise=true , address=45.91.116.9:4267 , piece_id=TTOGGN6SOWJGZO4RK5RRIROMOPAEJP77I3PD62PT3YE54R5Y4HLA, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:31.780226864 (257h44m26.211306995s) [noise=true , address=79.127.219.51:4215 , piece_id=VIF6J4NJVTL5PLLBK4BJEZN36CNFMLYS65F6JAIZFHA52CXOJCTQ, seg_pos={0 0}, upload=268006]: Uploading piece...
Aug 19 11:10:33.270967621 (257h44m27.702047742s) [upload=268006]: Next returned nil segment
Aug 19 11:10:33.270983160 (257h44m27.702063291s) [upload=268006]: Waiting for error group managing segments...
Aug 19 11:10:33.659531166 (257h44m28.090611297s) [noise=true , address=79.127.219.38:4027 , piece_id=CR4BD5LEHVB3DL2QO4G2HJ6ERIMKAPIYLQ5S5Z7L3ZYXI7K6U2XQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.701294538 (257h44m28.132374669s) [noise=true , address=104.36.82.60:4361 , piece_id=3XV3AZONK3457X5RDQG254NCUW4HE6HO76VFNKT5XSFPXX4FE2BQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.705700798 (257h44m28.136780919s) [noise=true , address=79.127.219.51:4215 , piece_id=VIF6J4NJVTL5PLLBK4BJEZN36CNFMLYS65F6JAIZFHA52CXOJCTQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.708143333 (257h44m28.139223464s) [noise=true , address=77.90.31.18:4808 , piece_id=4F6AMQMVBQXCQFLTXM5PCSUZZLH44DX5CQJ75CCLFQ573KBT5E5A, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.731526073 (257h44m28.162606194s) [noise=true , address=77.90.31.15:4709 , piece_id=4URFK2HRW43ED66DVTDQIXQOB7YFBTFVV5D6KFSDUL23KJFOJCYQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.762966121 (257h44m28.194046242s) [noise=true , address=77.90.31.17:4799 , piece_id=6R3TF3GHRKXNMCIALZNVLQVCRYCRJGNXYHEDWAXWONMLTNJDN3MA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.783901640 (257h44m28.214981761s) [noise=true , address=104.36.82.6:4908 , piece_id=YIPIUCCT5CILBJYX4QFL7SJXEEMUI4K56KZGW32PNO3UA75TKVJA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.790729604 (257h44m28.221809736s) [noise=true , address=45.91.116.6:4996 , piece_id=T4ZQG3LPULEXWBT2FUG4G5PSF5VB5HTIWN7F6JR6CMS3Q3XFQ6AA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.813267900 (257h44m28.244348021s) [noise=true , address=104.36.82.23:4470 , piece_id=AICIAJGD4NUYTNHNNZU3RY3QC6QZUUVBG33CX44URVJNRKSZEKZQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.815730592 (257h44m28.246810723s) [noise=true , address=45.91.116.5:4301 , piece_id=VGGQTH6ITD5SEFRDLDTU6APZI2APNPZ3Q5DJJRC6P2BTX237VTEA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.830686509 (257h44m28.26176664s) [noise=true , address=77.90.31.10:4538 , piece_id=W65SVEGF4OTV24Z6MGOPMXCE34WBLUNEWRRDADFPOQXBZVXGSODQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.840174453 (257h44m28.271254584s) [noise=true , address=45.91.116.9:4267 , piece_id=TTOGGN6SOWJGZO4RK5RRIROMOPAEJP77I3PD62PT3YE54R5Y4HLA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.840496980 (257h44m28.271577111s) [noise=true , address=45.91.116.14:4220 , piece_id=E6UPNYACCED4KHG6SI57N5FWUQ7NWPUD6YTHY63BEU7LOJ4HBKVA, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.867275459 (257h44m28.29835558s) [noise=true , address=45.91.116.12:4229 , piece_id=BCGJP7MCGW4ASHTAN7IGDZZA67IYV47R7IKTCXLHZJJEZNZPU7BQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: <nil>
Aug 19 11:10:33.867302751 (257h44m28.298382882s) [seg_pos={0 0}, upload=268006]: Segment reached optimal threshold of 14 pieces.
Aug 19 11:10:33.867471283 (257h44m28.298551404s) [noise=true , address=77.90.31.21:4903 , piece_id=OW2SO4EZYVXB7R6AWRZS2AZLBCN3TT7NOLIJATFQ6F6SNVGAY7IQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rZYWQ1a2sTKnFpF28iQVrem2s8nSpXkeJzZDaS7jvTZNANdfp): protocol: expected piece hash; context canceled; EOF
Aug 19 11:10:33.867516500 (257h44m28.298596631s) [noise=true , address=77.90.31.19:4876 , piece_id=KXYZROJUTF4C5CTPRLKWL6P4WLSIFP4BKBTOPUBOAVHTABMXKQ7Q, seg_pos={0 0}, upload=268006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12H3UNBGeAufYSdVvEgYkv2QWeFtjKMvHvYGeqGeBvjEHqDDjBB): protocol: expected piece hash; context canceled; EOF
Aug 19 11:10:33.867560314 (257h44m28.298640445s) [noise=true , address=45.91.116.13:4161 , piece_id=7X2RJXTNUGBAFHTDWKMXTZM3BPHCFGX3KPDOVTG4GRCKQ5C6GPPQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TaXapd6q5Vh1CQ5tQP98iQqVY99HFdGGv8CebGq49D9aKR9n2): protocol: expected piece hash; context canceled; EOF
Aug 19 11:10:33.867646239 (257h44m28.29872637s) [noise=true , address=77.90.31.13:4644 , piece_id=NOUOHAODVLQDH6AQHMAP7UCQ54Q235PODYSH6C254VDJGPKINGXQ, seg_pos={0 0}, upload=268006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13QCH91kRMQjVj2ASsVDcyXQ1aapQRVeUm6MMCGimsR6Ns973h): protocol: write tcp 188.166.225.170:32990->77.90.31.13:4644: use of closed network connection; write tcp 188.166.225.170:32990->77.90.31.13:4644: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32990->77.90.31.13:4644: use of closed network connection
Aug 19 11:10:33.867666487 (257h44m28.298746618s) [seg_pos={0 0}, upload=268006]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 11:10:33.867739918 (257h44m28.298820049s) [upload=268006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 11:10:33.867744586 (257h44m28.298824717s) [upload=268006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:10:34.562058994 (257h44m28.993139125s) [upload=268006]: Done uploading segments...
Aug 19 11:10:34.562077750 (257h44m28.993157881s) [upload=268006]: Upload finished. err: <nil>
Duration: 4.460160952s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
6.752928333s
FAIL (2025-08-19 11:10:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.752950957s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [...............X..X.......X........................X........] duration:
1.906009871s
FAIL (2025-08-19 11:09:52 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 1.906078543s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
5.903280692s
FAIL (2025-08-19 11:09:48 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 10:40:45.665627209 (257h14m40.09670734s) [upload=267489]: Starting upload
Aug 19 10:40:45.665699077 (257h14m40.096779208s) [upload=267489]: Done starting upload
Aug 19 10:40:45.665778629 (257h14m40.09685875s) [upload=267489]: Uploading segments...
Aug 19 10:40:45.665894090 (257h14m40.096974221s) [upload=267489]: Got next segment. Inline: false
Aug 19 10:40:45.665935800 (257h14m40.097015921s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 10:40:45.665940429 (257h14m40.09702056s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 10:40:46.709136118 (257h14m41.140216269s) [seg_pos={0 0}, upload=267489]: Begin upload segment...
Aug 19 10:40:46.709184060 (257h14m41.140264191s) [seg_pos={0 0}, upload=267489]: Done begin upload segment.
Aug 19 10:40:46.709190081 (257h14m41.140270212s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709202595 (257h14m41.140282726s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709681591 (257h14m41.140761732s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710102768 (257h14m41.141182909s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710390197 (257h14m41.141470328s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710763292 (257h14m41.141843423s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711131476 (257h14m41.142211597s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711510341 (257h14m41.142590472s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711832728 (257h14m41.142912859s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712238134 (257h14m41.143318275s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712581220 (257h14m41.143661351s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712930749 (257h14m41.14401089s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.713293623 (257h14m41.144373754s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715095551 (257h14m41.146175682s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715413158 (257h14m41.146493289s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715701190 (257h14m41.146781321s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715940758 (257h14m41.147020889s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.716184494 (257h14m41.147264625s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:47.857642100 (257h14m42.288722231s) [upload=267489]: Next returned nil segment
Aug 19 10:40:47.857661007 (257h14m42.288741138s) [upload=267489]: Waiting for error group managing segments...
Aug 19 10:40:48.143813827 (257h14m42.574893958s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.608965526 (257h14m43.040045668s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.638598507 (257h14m43.069678638s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.646238846 (257h14m43.077318977s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.688386852 (257h14m43.119466983s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.714878352 (257h14m43.145958483s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.716762747 (257h14m43.147842868s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.721383138 (257h14m43.152463269s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.741844700 (257h14m43.172924831s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.746036220 (257h14m43.177116351s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.761436808 (257h14m43.192516939s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.773758213 (257h14m43.204838344s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.780197735 (257h14m43.211277856s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798552525 (257h14m43.229632656s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798562594 (257h14m43.229642725s) [seg_pos={0 0}, upload=267489]: Segment reached optimal threshold of 14 pieces.
Aug 19 10:40:48.798678546 (257h14m43.229758677s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Li7UVDCVmainU4mkxtFyGky2wi6989Qu12BCQGgNn6roNX2A3): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798757777 (257h14m43.229837919s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHwFxcUjPsn12uHEAH3fBDJKv1v1tM8mRJXoFrZrG5djdswF5): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798830447 (257h14m43.229910738s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tvmo3nSgBPcFYW9MmZQGUeMoD3m3ykJHwVMj6tj69iQKgX3Mz): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798919998 (257h14m43.230000169s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ScTVpUJBg1xUCzmg1zKsNQ5AZgRbk65md6hgzst9QQ6Vgj4Fc): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798938083 (257h14m43.230018244s) [seg_pos={0 0}, upload=267489]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 10:40:48.799014489 (257h14m43.23009462s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 10:40:48.799018236 (257h14m43.230098367s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 10:40:49.423810646 (257h14m43.854890767s) [upload=267489]: Done uploading segments...
Aug 19 10:40:49.423827939 (257h14m43.85490807s) [upload=267489]: Upload finished. err: <nil>
Duration: 5.90331144s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
6.610435043s
FAIL (2025-08-19 11:09:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.610450412s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [................X..X.......X........................X.......] duration:
2.059898904s
FAIL (2025-08-19 11:09:31 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 2.059924745s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
6.645169846s
FAIL (2025-08-19 11:09:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.645186568s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
5.910313425s
FAIL (2025-08-19 11:09:18 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 10:40:45.665627209 (257h14m40.09670734s) [upload=267489]: Starting upload
Aug 19 10:40:45.665699077 (257h14m40.096779208s) [upload=267489]: Done starting upload
Aug 19 10:40:45.665778629 (257h14m40.09685875s) [upload=267489]: Uploading segments...
Aug 19 10:40:45.665894090 (257h14m40.096974221s) [upload=267489]: Got next segment. Inline: false
Aug 19 10:40:45.665935800 (257h14m40.097015921s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 10:40:45.665940429 (257h14m40.09702056s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 10:40:46.709136118 (257h14m41.140216269s) [seg_pos={0 0}, upload=267489]: Begin upload segment...
Aug 19 10:40:46.709184060 (257h14m41.140264191s) [seg_pos={0 0}, upload=267489]: Done begin upload segment.
Aug 19 10:40:46.709190081 (257h14m41.140270212s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709202595 (257h14m41.140282726s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709681591 (257h14m41.140761732s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710102768 (257h14m41.141182909s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710390197 (257h14m41.141470328s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710763292 (257h14m41.141843423s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711131476 (257h14m41.142211597s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711510341 (257h14m41.142590472s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711832728 (257h14m41.142912859s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712238134 (257h14m41.143318275s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712581220 (257h14m41.143661351s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712930749 (257h14m41.14401089s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.713293623 (257h14m41.144373754s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715095551 (257h14m41.146175682s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715413158 (257h14m41.146493289s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715701190 (257h14m41.146781321s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715940758 (257h14m41.147020889s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.716184494 (257h14m41.147264625s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:47.857642100 (257h14m42.288722231s) [upload=267489]: Next returned nil segment
Aug 19 10:40:47.857661007 (257h14m42.288741138s) [upload=267489]: Waiting for error group managing segments...
Aug 19 10:40:48.143813827 (257h14m42.574893958s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.608965526 (257h14m43.040045668s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.638598507 (257h14m43.069678638s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.646238846 (257h14m43.077318977s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.688386852 (257h14m43.119466983s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.714878352 (257h14m43.145958483s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.716762747 (257h14m43.147842868s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.721383138 (257h14m43.152463269s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.741844700 (257h14m43.172924831s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.746036220 (257h14m43.177116351s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.761436808 (257h14m43.192516939s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.773758213 (257h14m43.204838344s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.780197735 (257h14m43.211277856s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798552525 (257h14m43.229632656s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798562594 (257h14m43.229642725s) [seg_pos={0 0}, upload=267489]: Segment reached optimal threshold of 14 pieces.
Aug 19 10:40:48.798678546 (257h14m43.229758677s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Li7UVDCVmainU4mkxtFyGky2wi6989Qu12BCQGgNn6roNX2A3): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798757777 (257h14m43.229837919s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHwFxcUjPsn12uHEAH3fBDJKv1v1tM8mRJXoFrZrG5djdswF5): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798830447 (257h14m43.229910738s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tvmo3nSgBPcFYW9MmZQGUeMoD3m3ykJHwVMj6tj69iQKgX3Mz): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798919998 (257h14m43.230000169s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ScTVpUJBg1xUCzmg1zKsNQ5AZgRbk65md6hgzst9QQ6Vgj4Fc): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798938083 (257h14m43.230018244s) [seg_pos={0 0}, upload=267489]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 10:40:48.799014489 (257h14m43.23009462s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 10:40:48.799018236 (257h14m43.230098367s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 10:40:49.423810646 (257h14m43.854890767s) [upload=267489]: Done uploading segments...
Aug 19 10:40:49.423827939 (257h14m43.85490807s) [upload=267489]: Upload finished. err: <nil>
Duration: 5.910338031s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.................X..X.......X........................X......] duration:
1.909537994s
FAIL (2025-08-19 11:08:36 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 1.909562011s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
6.760611061s
FAIL (2025-08-19 11:08:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.760672578s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [X......................X....................................] duration:
5.702023405s
FAIL (2025-08-19 11:08:20 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 10:40:45.665627209 (257h14m40.09670734s) [upload=267489]: Starting upload
Aug 19 10:40:45.665699077 (257h14m40.096779208s) [upload=267489]: Done starting upload
Aug 19 10:40:45.665778629 (257h14m40.09685875s) [upload=267489]: Uploading segments...
Aug 19 10:40:45.665894090 (257h14m40.096974221s) [upload=267489]: Got next segment. Inline: false
Aug 19 10:40:45.665935800 (257h14m40.097015921s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 10:40:45.665940429 (257h14m40.09702056s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 10:40:46.709136118 (257h14m41.140216269s) [seg_pos={0 0}, upload=267489]: Begin upload segment...
Aug 19 10:40:46.709184060 (257h14m41.140264191s) [seg_pos={0 0}, upload=267489]: Done begin upload segment.
Aug 19 10:40:46.709190081 (257h14m41.140270212s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709202595 (257h14m41.140282726s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709681591 (257h14m41.140761732s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710102768 (257h14m41.141182909s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710390197 (257h14m41.141470328s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710763292 (257h14m41.141843423s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711131476 (257h14m41.142211597s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711510341 (257h14m41.142590472s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711832728 (257h14m41.142912859s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712238134 (257h14m41.143318275s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712581220 (257h14m41.143661351s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712930749 (257h14m41.14401089s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.713293623 (257h14m41.144373754s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715095551 (257h14m41.146175682s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715413158 (257h14m41.146493289s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715701190 (257h14m41.146781321s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715940758 (257h14m41.147020889s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.716184494 (257h14m41.147264625s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:47.857642100 (257h14m42.288722231s) [upload=267489]: Next returned nil segment
Aug 19 10:40:47.857661007 (257h14m42.288741138s) [upload=267489]: Waiting for error group managing segments...
Aug 19 10:40:48.143813827 (257h14m42.574893958s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.608965526 (257h14m43.040045668s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.638598507 (257h14m43.069678638s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.646238846 (257h14m43.077318977s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.688386852 (257h14m43.119466983s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.714878352 (257h14m43.145958483s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.716762747 (257h14m43.147842868s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.721383138 (257h14m43.152463269s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.741844700 (257h14m43.172924831s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.746036220 (257h14m43.177116351s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.761436808 (257h14m43.192516939s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.773758213 (257h14m43.204838344s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.780197735 (257h14m43.211277856s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798552525 (257h14m43.229632656s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798562594 (257h14m43.229642725s) [seg_pos={0 0}, upload=267489]: Segment reached optimal threshold of 14 pieces.
Aug 19 10:40:48.798678546 (257h14m43.229758677s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Li7UVDCVmainU4mkxtFyGky2wi6989Qu12BCQGgNn6roNX2A3): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798757777 (257h14m43.229837919s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHwFxcUjPsn12uHEAH3fBDJKv1v1tM8mRJXoFrZrG5djdswF5): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798830447 (257h14m43.229910738s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tvmo3nSgBPcFYW9MmZQGUeMoD3m3ykJHwVMj6tj69iQKgX3Mz): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798919998 (257h14m43.230000169s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ScTVpUJBg1xUCzmg1zKsNQ5AZgRbk65md6hgzst9QQ6Vgj4Fc): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798938083 (257h14m43.230018244s) [seg_pos={0 0}, upload=267489]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 10:40:48.799014489 (257h14m43.23009462s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 10:40:48.799018236 (257h14m43.230098367s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 10:40:49.423810646 (257h14m43.854890767s) [upload=267489]: Done uploading segments...
Aug 19 10:40:49.423827939 (257h14m43.85490807s) [upload=267489]: Upload finished. err: <nil>
Duration: 5.702043645s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
7.018268248s
FAIL (2025-08-19 11:08:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.018289628s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..................X..X.......X........................X.....] duration:
1.842914534s
FAIL (2025-08-19 11:08:02 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 1.842942687s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.X......................X...................................] duration:
5.932968934s
FAIL (2025-08-19 11:07:38 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 10:40:45.665627209 (257h14m40.09670734s) [upload=267489]: Starting upload
Aug 19 10:40:45.665699077 (257h14m40.096779208s) [upload=267489]: Done starting upload
Aug 19 10:40:45.665778629 (257h14m40.09685875s) [upload=267489]: Uploading segments...
Aug 19 10:40:45.665894090 (257h14m40.096974221s) [upload=267489]: Got next segment. Inline: false
Aug 19 10:40:45.665935800 (257h14m40.097015921s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 10:40:45.665940429 (257h14m40.09702056s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 10:40:46.709136118 (257h14m41.140216269s) [seg_pos={0 0}, upload=267489]: Begin upload segment...
Aug 19 10:40:46.709184060 (257h14m41.140264191s) [seg_pos={0 0}, upload=267489]: Done begin upload segment.
Aug 19 10:40:46.709190081 (257h14m41.140270212s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709202595 (257h14m41.140282726s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709681591 (257h14m41.140761732s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710102768 (257h14m41.141182909s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710390197 (257h14m41.141470328s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710763292 (257h14m41.141843423s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711131476 (257h14m41.142211597s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711510341 (257h14m41.142590472s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711832728 (257h14m41.142912859s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712238134 (257h14m41.143318275s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712581220 (257h14m41.143661351s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712930749 (257h14m41.14401089s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.713293623 (257h14m41.144373754s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715095551 (257h14m41.146175682s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715413158 (257h14m41.146493289s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715701190 (257h14m41.146781321s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715940758 (257h14m41.147020889s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.716184494 (257h14m41.147264625s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:47.857642100 (257h14m42.288722231s) [upload=267489]: Next returned nil segment
Aug 19 10:40:47.857661007 (257h14m42.288741138s) [upload=267489]: Waiting for error group managing segments...
Aug 19 10:40:48.143813827 (257h14m42.574893958s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.608965526 (257h14m43.040045668s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.638598507 (257h14m43.069678638s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.646238846 (257h14m43.077318977s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.688386852 (257h14m43.119466983s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.714878352 (257h14m43.145958483s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.716762747 (257h14m43.147842868s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.721383138 (257h14m43.152463269s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.741844700 (257h14m43.172924831s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.746036220 (257h14m43.177116351s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.761436808 (257h14m43.192516939s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.773758213 (257h14m43.204838344s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.780197735 (257h14m43.211277856s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798552525 (257h14m43.229632656s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798562594 (257h14m43.229642725s) [seg_pos={0 0}, upload=267489]: Segment reached optimal threshold of 14 pieces.
Aug 19 10:40:48.798678546 (257h14m43.229758677s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Li7UVDCVmainU4mkxtFyGky2wi6989Qu12BCQGgNn6roNX2A3): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798757777 (257h14m43.229837919s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHwFxcUjPsn12uHEAH3fBDJKv1v1tM8mRJXoFrZrG5djdswF5): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798830447 (257h14m43.229910738s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tvmo3nSgBPcFYW9MmZQGUeMoD3m3ykJHwVMj6tj69iQKgX3Mz): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798919998 (257h14m43.230000169s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ScTVpUJBg1xUCzmg1zKsNQ5AZgRbk65md6hgzst9QQ6Vgj4Fc): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798938083 (257h14m43.230018244s) [seg_pos={0 0}, upload=267489]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 10:40:48.799014489 (257h14m43.23009462s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 10:40:48.799018236 (257h14m43.230098367s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 10:40:49.423810646 (257h14m43.854890767s) [upload=267489]: Done uploading segments...
Aug 19 10:40:49.423827939 (257h14m43.85490807s) [upload=267489]: Upload finished. err: <nil>
Duration: 5.932998781s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [...................X..X.......X........................X....] duration:
1.842227226s
FAIL (2025-08-19 11:07:11 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 1.84225469s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
6.547498861s
FAIL (2025-08-19 11:07:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.547614692s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [....................X..X.......X........................X...] duration:
1.844453248s
FAIL (2025-08-19 11:06:52 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 1.84453288s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X......................X..................................] duration:
6.401249504s
FAIL (2025-08-19 11:06:49 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 10:40:45.665627209 (257h14m40.09670734s) [upload=267489]: Starting upload
Aug 19 10:40:45.665699077 (257h14m40.096779208s) [upload=267489]: Done starting upload
Aug 19 10:40:45.665778629 (257h14m40.09685875s) [upload=267489]: Uploading segments...
Aug 19 10:40:45.665894090 (257h14m40.096974221s) [upload=267489]: Got next segment. Inline: false
Aug 19 10:40:45.665935800 (257h14m40.097015921s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 10:40:45.665940429 (257h14m40.09702056s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 10:40:46.709136118 (257h14m41.140216269s) [seg_pos={0 0}, upload=267489]: Begin upload segment...
Aug 19 10:40:46.709184060 (257h14m41.140264191s) [seg_pos={0 0}, upload=267489]: Done begin upload segment.
Aug 19 10:40:46.709190081 (257h14m41.140270212s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709202595 (257h14m41.140282726s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709681591 (257h14m41.140761732s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710102768 (257h14m41.141182909s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710390197 (257h14m41.141470328s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710763292 (257h14m41.141843423s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711131476 (257h14m41.142211597s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711510341 (257h14m41.142590472s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711832728 (257h14m41.142912859s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712238134 (257h14m41.143318275s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712581220 (257h14m41.143661351s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712930749 (257h14m41.14401089s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.713293623 (257h14m41.144373754s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715095551 (257h14m41.146175682s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715413158 (257h14m41.146493289s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715701190 (257h14m41.146781321s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715940758 (257h14m41.147020889s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.716184494 (257h14m41.147264625s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:47.857642100 (257h14m42.288722231s) [upload=267489]: Next returned nil segment
Aug 19 10:40:47.857661007 (257h14m42.288741138s) [upload=267489]: Waiting for error group managing segments...
Aug 19 10:40:48.143813827 (257h14m42.574893958s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.608965526 (257h14m43.040045668s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.638598507 (257h14m43.069678638s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.646238846 (257h14m43.077318977s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.688386852 (257h14m43.119466983s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.714878352 (257h14m43.145958483s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.716762747 (257h14m43.147842868s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.721383138 (257h14m43.152463269s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.741844700 (257h14m43.172924831s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.746036220 (257h14m43.177116351s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.761436808 (257h14m43.192516939s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.773758213 (257h14m43.204838344s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.780197735 (257h14m43.211277856s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798552525 (257h14m43.229632656s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798562594 (257h14m43.229642725s) [seg_pos={0 0}, upload=267489]: Segment reached optimal threshold of 14 pieces.
Aug 19 10:40:48.798678546 (257h14m43.229758677s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Li7UVDCVmainU4mkxtFyGky2wi6989Qu12BCQGgNn6roNX2A3): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798757777 (257h14m43.229837919s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHwFxcUjPsn12uHEAH3fBDJKv1v1tM8mRJXoFrZrG5djdswF5): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798830447 (257h14m43.229910738s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tvmo3nSgBPcFYW9MmZQGUeMoD3m3ykJHwVMj6tj69iQKgX3Mz): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798919998 (257h14m43.230000169s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ScTVpUJBg1xUCzmg1zKsNQ5AZgRbk65md6hgzst9QQ6Vgj4Fc): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798938083 (257h14m43.230018244s) [seg_pos={0 0}, upload=267489]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 10:40:48.799014489 (257h14m43.23009462s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 10:40:48.799018236 (257h14m43.230098367s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 10:40:49.423810646 (257h14m43.854890767s) [upload=267489]: Done uploading segments...
Aug 19 10:40:49.423827939 (257h14m43.85490807s) [upload=267489]: Upload finished. err: <nil>
Duration: 6.401282798s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
6.333323393s
FAIL (2025-08-19 11:06:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.333345025s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [.....................X..X.......X........................X..] duration:
2.4050525s
FAIL (2025-08-19 11:06:20 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 2.405081197s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [......................X..X.......X........................X.] duration:
2.472104648s
FAIL (2025-08-19 11:05:42 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 19 11:04:30.217546646 (257h38m24.648626777s) [upload=267896]: Starting upload
Aug 19 11:04:30.217591562 (257h38m24.648671693s) [upload=267896]: Done starting upload
Aug 19 11:04:30.217635435 (257h38m24.648715566s) [upload=267896]: Uploading segments...
Aug 19 11:04:30.217669160 (257h38m24.648749281s) [upload=267896]: Got next segment. Inline: true
Aug 19 11:04:30.217672447 (257h38m24.648752578s) [upload=267896]: Waiting for error group managing segments...
Aug 19 11:04:30.217702484 (257h38m24.648782605s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 11:04:30.217706241 (257h38m24.648786372s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 19 11:04:30.217708596 (257h38m24.648788717s) [upload=267896]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 11:04:30.655984702 (257h38m25.087064833s) [upload=267896]: Done uploading segments...
Aug 19 11:04:30.656008768 (257h38m25.087088929s) [upload=267896]: Upload finished. err: <nil>
Duration: 2.472125167s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...X......................X.................................] duration:
5.682381617s
FAIL (2025-08-19 11:05:36 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 10:40:45.665627209 (257h14m40.09670734s) [upload=267489]: Starting upload
Aug 19 10:40:45.665699077 (257h14m40.096779208s) [upload=267489]: Done starting upload
Aug 19 10:40:45.665778629 (257h14m40.09685875s) [upload=267489]: Uploading segments...
Aug 19 10:40:45.665894090 (257h14m40.096974221s) [upload=267489]: Got next segment. Inline: false
Aug 19 10:40:45.665935800 (257h14m40.097015921s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 10:40:45.665940429 (257h14m40.09702056s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 10:40:46.709136118 (257h14m41.140216269s) [seg_pos={0 0}, upload=267489]: Begin upload segment...
Aug 19 10:40:46.709184060 (257h14m41.140264191s) [seg_pos={0 0}, upload=267489]: Done begin upload segment.
Aug 19 10:40:46.709190081 (257h14m41.140270212s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709202595 (257h14m41.140282726s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709681591 (257h14m41.140761732s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710102768 (257h14m41.141182909s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710390197 (257h14m41.141470328s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710763292 (257h14m41.141843423s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711131476 (257h14m41.142211597s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711510341 (257h14m41.142590472s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711832728 (257h14m41.142912859s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712238134 (257h14m41.143318275s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712581220 (257h14m41.143661351s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712930749 (257h14m41.14401089s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.713293623 (257h14m41.144373754s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715095551 (257h14m41.146175682s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715413158 (257h14m41.146493289s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715701190 (257h14m41.146781321s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715940758 (257h14m41.147020889s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.716184494 (257h14m41.147264625s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:47.857642100 (257h14m42.288722231s) [upload=267489]: Next returned nil segment
Aug 19 10:40:47.857661007 (257h14m42.288741138s) [upload=267489]: Waiting for error group managing segments...
Aug 19 10:40:48.143813827 (257h14m42.574893958s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.608965526 (257h14m43.040045668s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.638598507 (257h14m43.069678638s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.646238846 (257h14m43.077318977s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.688386852 (257h14m43.119466983s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.714878352 (257h14m43.145958483s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.716762747 (257h14m43.147842868s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.721383138 (257h14m43.152463269s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.741844700 (257h14m43.172924831s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.746036220 (257h14m43.177116351s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.761436808 (257h14m43.192516939s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.773758213 (257h14m43.204838344s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.780197735 (257h14m43.211277856s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798552525 (257h14m43.229632656s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798562594 (257h14m43.229642725s) [seg_pos={0 0}, upload=267489]: Segment reached optimal threshold of 14 pieces.
Aug 19 10:40:48.798678546 (257h14m43.229758677s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Li7UVDCVmainU4mkxtFyGky2wi6989Qu12BCQGgNn6roNX2A3): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798757777 (257h14m43.229837919s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHwFxcUjPsn12uHEAH3fBDJKv1v1tM8mRJXoFrZrG5djdswF5): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798830447 (257h14m43.229910738s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tvmo3nSgBPcFYW9MmZQGUeMoD3m3ykJHwVMj6tj69iQKgX3Mz): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798919998 (257h14m43.230000169s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ScTVpUJBg1xUCzmg1zKsNQ5AZgRbk65md6hgzst9QQ6Vgj4Fc): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798938083 (257h14m43.230018244s) [seg_pos={0 0}, upload=267489]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 10:40:48.799014489 (257h14m43.23009462s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 10:40:48.799018236 (257h14m43.230098367s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 10:40:49.423810646 (257h14m43.854890767s) [upload=267489]: Done uploading segments...
Aug 19 10:40:49.423827939 (257h14m43.85490807s) [upload=267489]: Upload finished. err: <nil>
Duration: 5.682412036s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
6.643324186s
FAIL (2025-08-19 11:05:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.643350668s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
3 out of 60 is failed (threshold=3) [X...X......................X................................] duration:
5.851955085s
FAIL (2025-08-19 11:04:59 UTC)
orbiter warning:
operation: ListObjects
======ERROR======
unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M
======LOGS======
Aug 19 10:40:45.665627209 (257h14m40.09670734s) [upload=267489]: Starting upload
Aug 19 10:40:45.665699077 (257h14m40.096779208s) [upload=267489]: Done starting upload
Aug 19 10:40:45.665778629 (257h14m40.09685875s) [upload=267489]: Uploading segments...
Aug 19 10:40:45.665894090 (257h14m40.096974221s) [upload=267489]: Got next segment. Inline: false
Aug 19 10:40:45.665935800 (257h14m40.097015921s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 19 10:40:45.665940429 (257h14m40.09702056s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 19 10:40:46.709136118 (257h14m41.140216269s) [seg_pos={0 0}, upload=267489]: Begin upload segment...
Aug 19 10:40:46.709184060 (257h14m41.140264191s) [seg_pos={0 0}, upload=267489]: Done begin upload segment.
Aug 19 10:40:46.709190081 (257h14m41.140270212s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709202595 (257h14m41.140282726s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.709681591 (257h14m41.140761732s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710102768 (257h14m41.141182909s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710390197 (257h14m41.141470328s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.710763292 (257h14m41.141843423s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711131476 (257h14m41.142211597s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711510341 (257h14m41.142590472s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.711832728 (257h14m41.142912859s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712238134 (257h14m41.143318275s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712581220 (257h14m41.143661351s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.712930749 (257h14m41.14401089s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.713293623 (257h14m41.144373754s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715095551 (257h14m41.146175682s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715413158 (257h14m41.146493289s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715701190 (257h14m41.146781321s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.715940758 (257h14m41.147020889s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:46.716184494 (257h14m41.147264625s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Uploading piece...
Aug 19 10:40:47.857642100 (257h14m42.288722231s) [upload=267489]: Next returned nil segment
Aug 19 10:40:47.857661007 (257h14m42.288741138s) [upload=267489]: Waiting for error group managing segments...
Aug 19 10:40:48.143813827 (257h14m42.574893958s) [noise=true , address=66.23.193.175:4561 , piece_id=H3PJNOZM342ESCWBAAMHGXFOOHASVQP4FLRHG7N3PRKEKZUFEZTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.608965526 (257h14m43.040045668s) [noise=true , address=45.91.116.11:4141 , piece_id=X5RSKA63FDNT35FLE4YS6MCYV4L33GIA55NP667YN33U424OTGAQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.638598507 (257h14m43.069678638s) [noise=true , address=45.91.116.5:4317 , piece_id=YAJ4I42W2QYJZAO2JB55GBDKYBO2EHWXVK5KC46BGGTGML7AMZOQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.646238846 (257h14m43.077318977s) [noise=true , address=104.36.82.3:4776 , piece_id=QALJA3ULXHEVBDBMKNDZE2DBANPEW3FVUTXPV62AJVTIBRVESDBA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.688386852 (257h14m43.119466983s) [noise=true , address=79.127.219.38:4027 , piece_id=OWOQ32DXV2KILPD77JTXLFAAX77ANGEKNFQ2IFNZZPQBQJU3Y6WA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.714878352 (257h14m43.145958483s) [noise=true , address=45.91.116.9:4279 , piece_id=3F4ITIXIQI4XTH63P2MP52BVMWUZK7QV6SG6BIJJIIELTELWGVTA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.716762747 (257h14m43.147842868s) [noise=true , address=77.90.31.21:4894 , piece_id=WW4NSJESIP2DPSH432P5GSFQNM4XN4PVUKMUGSRSTUHOJPRY2F6Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.721383138 (257h14m43.152463269s) [noise=true , address=45.91.116.13:4157 , piece_id=QISOR7HDO5NRUHUY6EFMKGIWPZHXJKHMGEGDDYCSRW24O5BHTTLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.741844700 (257h14m43.172924831s) [noise=true , address=79.127.201.215:4353 , piece_id=2MUDZKQ3TLVMENMHPJRAQWVWFPMWB6VZZR6DH4QDTMN5OHY2QLEA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.746036220 (257h14m43.177116351s) [noise=true , address=104.36.82.6:4908 , piece_id=NQRSW2LJGBXXXAGGHWSWOMW2DBOE7WP7WLH3FWFPXMWCANROVRXA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.761436808 (257h14m43.192516939s) [noise=true , address=77.90.31.16:4753 , piece_id=XEJFJRPBNBEMPHWI4J3AFPZEQUS76HEPMWCXH6G6A3F7ZQ5ZQPUA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.773758213 (257h14m43.204838344s) [noise=true , address=77.90.31.17:4778 , piece_id=LDBTWH54VBU65ZG5SRNOCKOEFLHDB4HACG645B2Q42AIYPXWUHNQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.780197735 (257h14m43.211277856s) [noise=true , address=104.36.82.23:4458 , piece_id=LNCOVMZT2E3WIH6O6BEPVP52LHHOHNKE63F5XEPCP5NAU6CJ554A, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798552525 (257h14m43.229632656s) [noise=true , address=104.36.82.19:4435 , piece_id=GRZC7YCBLTRX3FZ7AUGORBO32NHSP5U4PANNNANIETCVMWE4ZU5Q, seg_pos={0 0}, upload=267489]: Done uploading piece. err: <nil>
Aug 19 10:40:48.798562594 (257h14m43.229642725s) [seg_pos={0 0}, upload=267489]: Segment reached optimal threshold of 14 pieces.
Aug 19 10:40:48.798678546 (257h14m43.229758677s) [noise=true , address=77.90.31.12:4598 , piece_id=QOK2Q6QQ4JQEVPPOG7SSJKTGEGQP4WCLHDF4Z45FGAI37C6PANHQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Li7UVDCVmainU4mkxtFyGky2wi6989Qu12BCQGgNn6roNX2A3): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798757777 (257h14m43.229837919s) [noise=true , address=104.36.82.44:4892 , piece_id=XKPY7MPBAGFTUUZSZBBHMYPZIR3H6VHQQU2LA5YA5Z2UHFDQIKLQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RHwFxcUjPsn12uHEAH3fBDJKv1v1tM8mRJXoFrZrG5djdswF5): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798830447 (257h14m43.229910738s) [noise=true , address=77.90.31.19:4862 , piece_id=F2ZG2JCOCAPPWLFS24EENLM6HSCJESS6AHYVAM5SEMFWXR74W4XA, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tvmo3nSgBPcFYW9MmZQGUeMoD3m3ykJHwVMj6tj69iQKgX3Mz): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798919998 (257h14m43.230000169s) [noise=true , address=77.90.31.13:4659 , piece_id=LA25EOMZC55MY6AEWQI6UJWHSD2OM2TAXFYLVJAEU462XLDIJYTQ, seg_pos={0 0}, upload=267489]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ScTVpUJBg1xUCzmg1zKsNQ5AZgRbk65md6hgzst9QQ6Vgj4Fc): protocol: expected piece hash; context canceled; EOF
Aug 19 10:40:48.798938083 (257h14m43.230018244s) [seg_pos={0 0}, upload=267489]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 19 10:40:48.799014489 (257h14m43.23009462s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 19 10:40:48.799018236 (257h14m43.230098367s) [upload=267489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 19 10:40:49.423810646 (257h14m43.854890767s) [upload=267489]: Done uploading segments...
Aug 19 10:40:49.423827939 (257h14m43.85490807s) [upload=267489]: Upload finished. err: <nil>
Duration: 5.851984602s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|