Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........X.......................................X.........] duration:
6.463114377s
FAIL (2025-08-09 00:32:46 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:25:21.054812319 (6h59m15.48589246s) [upload=7311]: Starting upload Aug 9 00:25:21.054887373 (6h59m15.485967514s) [upload=7311]: Done starting upload Aug 9 00:25:21.054974740 (6h59m15.486054871s) [upload=7311]: Uploading segments... Aug 9 00:25:21.055143473 (6h59m15.486223604s) [upload=7311]: Got next segment. Inline: false Aug 9 00:25:21.055207194 (6h59m15.486287325s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 00:25:21.055213838 (6h59m15.486293969s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 9 00:25:21.326628933 (6h59m15.757709064s) [upload=7311]: Next segment err: metaclient: Too Many Requests Aug 9 00:25:21.326662107 (6h59m15.757742238s) [upload=7311]: Done uploading segments... Aug 9 00:25:21.326682726 (6h59m15.757762907s) [upload=7311]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.463144293s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
746.37911ms
FAIL (2025-08-09 00:32:38 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 746.412213ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
6 out of 60 is failed (threshold=3) [...X........X..........X....X.....X.X.......................] duration:
2.001862071s
FAIL (2025-08-09 00:32:31 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 9 00:15:20.946114856 (6h49m15.377194997s) [upload=7145]: Starting upload
Aug 9 00:15:20.946168780 (6h49m15.377248911s) [upload=7145]: Done starting upload
Aug 9 00:15:20.946228393 (6h49m15.377308525s) [upload=7145]: Uploading segments...
Aug 9 00:15:20.946267749 (6h49m15.37734788s) [upload=7145]: Got next segment. Inline: true
Aug 9 00:15:20.946271877 (6h49m15.377352008s) [upload=7145]: Waiting for error group managing segments...
Aug 9 00:15:20.946306713 (6h49m15.377386845s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 00:15:20.946310951 (6h49m15.377391082s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 9 00:15:20.946313546 (6h49m15.377393677s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 00:15:21.403671645 (6h49m15.834751776s) [upload=7145]: Done uploading segments...
Aug 9 00:15:21.403691763 (6h49m15.834771894s) [upload=7145]: Upload finished. err: <nil>
Duration: 2.001889805s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
2.364631884s
FAIL (2025-08-09 00:32:16 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 2.364660167s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........X.......................................X........] duration:
6.951226185s
FAIL (2025-08-09 00:32:14 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:25:21.054812319 (6h59m15.48589246s) [upload=7311]: Starting upload Aug 9 00:25:21.054887373 (6h59m15.485967514s) [upload=7311]: Done starting upload Aug 9 00:25:21.054974740 (6h59m15.486054871s) [upload=7311]: Uploading segments... Aug 9 00:25:21.055143473 (6h59m15.486223604s) [upload=7311]: Got next segment. Inline: false Aug 9 00:25:21.055207194 (6h59m15.486287325s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 00:25:21.055213838 (6h59m15.486293969s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 9 00:25:21.326628933 (6h59m15.757709064s) [upload=7311]: Next segment err: metaclient: Too Many Requests Aug 9 00:25:21.326662107 (6h59m15.757742238s) [upload=7311]: Done uploading segments... Aug 9 00:25:21.326682726 (6h59m15.757762907s) [upload=7311]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.951302582s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
808.687814ms
FAIL (2025-08-09 00:31:43 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 808.722811ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
6 out of 60 is failed (threshold=3) [....X........X..........X....X.....X.X......................] duration:
1.896374414s
FAIL (2025-08-09 00:31:38 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 9 00:15:20.946114856 (6h49m15.377194997s) [upload=7145]: Starting upload
Aug 9 00:15:20.946168780 (6h49m15.377248911s) [upload=7145]: Done starting upload
Aug 9 00:15:20.946228393 (6h49m15.377308525s) [upload=7145]: Uploading segments...
Aug 9 00:15:20.946267749 (6h49m15.37734788s) [upload=7145]: Got next segment. Inline: true
Aug 9 00:15:20.946271877 (6h49m15.377352008s) [upload=7145]: Waiting for error group managing segments...
Aug 9 00:15:20.946306713 (6h49m15.377386845s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 00:15:20.946310951 (6h49m15.377391082s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 9 00:15:20.946313546 (6h49m15.377393677s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 00:15:21.403671645 (6h49m15.834751776s) [upload=7145]: Done uploading segments...
Aug 9 00:15:21.403691763 (6h49m15.834771894s) [upload=7145]: Upload finished. err: <nil>
Duration: 1.896403449s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [............X.......................................X.......] duration:
10.393690345s
FAIL (2025-08-09 00:31:14 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:25:21.054812319 (6h59m15.48589246s) [upload=7311]: Starting upload Aug 9 00:25:21.054887373 (6h59m15.485967514s) [upload=7311]: Done starting upload Aug 9 00:25:21.054974740 (6h59m15.486054871s) [upload=7311]: Uploading segments... Aug 9 00:25:21.055143473 (6h59m15.486223604s) [upload=7311]: Got next segment. Inline: false Aug 9 00:25:21.055207194 (6h59m15.486287325s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 00:25:21.055213838 (6h59m15.486293969s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 9 00:25:21.326628933 (6h59m15.757709064s) [upload=7311]: Next segment err: metaclient: Too Many Requests Aug 9 00:25:21.326662107 (6h59m15.757742238s) [upload=7311]: Done uploading segments... Aug 9 00:25:21.326682726 (6h59m15.757762907s) [upload=7311]: Upload finished. err: metaclient: Too Many Requests
Duration: 10.393725142s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
2.2854832s
FAIL (2025-08-09 00:31:10 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 2.285497748s
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
714.744954ms
FAIL (2025-08-09 00:31:02 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 714.77423ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
6 out of 60 is failed (threshold=3) [.....X........X..........X....X.....X.X.....................] duration:
1.875839206s
FAIL (2025-08-09 00:30:39 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 9 00:15:20.946114856 (6h49m15.377194997s) [upload=7145]: Starting upload
Aug 9 00:15:20.946168780 (6h49m15.377248911s) [upload=7145]: Done starting upload
Aug 9 00:15:20.946228393 (6h49m15.377308525s) [upload=7145]: Uploading segments...
Aug 9 00:15:20.946267749 (6h49m15.37734788s) [upload=7145]: Got next segment. Inline: true
Aug 9 00:15:20.946271877 (6h49m15.377352008s) [upload=7145]: Waiting for error group managing segments...
Aug 9 00:15:20.946306713 (6h49m15.377386845s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 00:15:20.946310951 (6h49m15.377391082s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 9 00:15:20.946313546 (6h49m15.377393677s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 00:15:21.403671645 (6h49m15.834751776s) [upload=7145]: Done uploading segments...
Aug 9 00:15:21.403691763 (6h49m15.834771894s) [upload=7145]: Upload finished. err: <nil>
Duration: 1.87586777s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............X.......................................X......] duration:
6.295265422s
FAIL (2025-08-09 00:30:22 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:25:21.054812319 (6h59m15.48589246s) [upload=7311]: Starting upload Aug 9 00:25:21.054887373 (6h59m15.485967514s) [upload=7311]: Done starting upload Aug 9 00:25:21.054974740 (6h59m15.486054871s) [upload=7311]: Uploading segments... Aug 9 00:25:21.055143473 (6h59m15.486223604s) [upload=7311]: Got next segment. Inline: false Aug 9 00:25:21.055207194 (6h59m15.486287325s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 00:25:21.055213838 (6h59m15.486293969s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 9 00:25:21.326628933 (6h59m15.757709064s) [upload=7311]: Next segment err: metaclient: Too Many Requests Aug 9 00:25:21.326662107 (6h59m15.757742238s) [upload=7311]: Done uploading segments... Aug 9 00:25:21.326682726 (6h59m15.757762907s) [upload=7311]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.295295701s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
720.155055ms
FAIL (2025-08-09 00:30:07 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 720.173871ms
edge
authservice
http
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.349009774s
FAIL (2025-08-09 00:30:05 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 2.349035413s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............X.......................................X.....] duration:
6.03359467s
FAIL (2025-08-09 00:29:54 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:25:21.054812319 (6h59m15.48589246s) [upload=7311]: Starting upload Aug 9 00:25:21.054887373 (6h59m15.485967514s) [upload=7311]: Done starting upload Aug 9 00:25:21.054974740 (6h59m15.486054871s) [upload=7311]: Uploading segments... Aug 9 00:25:21.055143473 (6h59m15.486223604s) [upload=7311]: Got next segment. Inline: false Aug 9 00:25:21.055207194 (6h59m15.486287325s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 00:25:21.055213838 (6h59m15.486293969s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 9 00:25:21.326628933 (6h59m15.757709064s) [upload=7311]: Next segment err: metaclient: Too Many Requests Aug 9 00:25:21.326662107 (6h59m15.757742238s) [upload=7311]: Done uploading segments... Aug 9 00:25:21.326682726 (6h59m15.757762907s) [upload=7311]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.033628945s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
739.310956ms
FAIL (2025-08-09 00:29:44 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 739.348388ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
6 out of 60 is failed (threshold=3) [......X........X..........X....X.....X.X....................] duration:
2.015876575s
FAIL (2025-08-09 00:29:42 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 9 00:15:20.946114856 (6h49m15.377194997s) [upload=7145]: Starting upload
Aug 9 00:15:20.946168780 (6h49m15.377248911s) [upload=7145]: Done starting upload
Aug 9 00:15:20.946228393 (6h49m15.377308525s) [upload=7145]: Uploading segments...
Aug 9 00:15:20.946267749 (6h49m15.37734788s) [upload=7145]: Got next segment. Inline: true
Aug 9 00:15:20.946271877 (6h49m15.377352008s) [upload=7145]: Waiting for error group managing segments...
Aug 9 00:15:20.946306713 (6h49m15.377386845s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 00:15:20.946310951 (6h49m15.377391082s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 9 00:15:20.946313546 (6h49m15.377393677s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 00:15:21.403671645 (6h49m15.834751776s) [upload=7145]: Done uploading segments...
Aug 9 00:15:21.403691763 (6h49m15.834771894s) [upload=7145]: Upload finished. err: <nil>
Duration: 2.015902003s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.383377078s
FAIL (2025-08-09 00:29:28 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 2.383402197s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
6 out of 60 is failed (threshold=3) [.......X........X..........X....X.....X.X...................] duration:
1.850765368s
FAIL (2025-08-09 00:29:08 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 9 00:15:20.946114856 (6h49m15.377194997s) [upload=7145]: Starting upload
Aug 9 00:15:20.946168780 (6h49m15.377248911s) [upload=7145]: Done starting upload
Aug 9 00:15:20.946228393 (6h49m15.377308525s) [upload=7145]: Uploading segments...
Aug 9 00:15:20.946267749 (6h49m15.37734788s) [upload=7145]: Got next segment. Inline: true
Aug 9 00:15:20.946271877 (6h49m15.377352008s) [upload=7145]: Waiting for error group managing segments...
Aug 9 00:15:20.946306713 (6h49m15.377386845s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 00:15:20.946310951 (6h49m15.377391082s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 9 00:15:20.946313546 (6h49m15.377393677s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 00:15:21.403671645 (6h49m15.834751776s) [upload=7145]: Done uploading segments...
Aug 9 00:15:21.403691763 (6h49m15.834771894s) [upload=7145]: Upload finished. err: <nil>
Duration: 1.850801128s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
730.829558ms
FAIL (2025-08-09 00:28:55 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 730.861819ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............X.......................................X....] duration:
7.492539003s
FAIL (2025-08-09 00:28:54 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:25:21.054812319 (6h59m15.48589246s) [upload=7311]: Starting upload Aug 9 00:25:21.054887373 (6h59m15.485967514s) [upload=7311]: Done starting upload Aug 9 00:25:21.054974740 (6h59m15.486054871s) [upload=7311]: Uploading segments... Aug 9 00:25:21.055143473 (6h59m15.486223604s) [upload=7311]: Got next segment. Inline: false Aug 9 00:25:21.055207194 (6h59m15.486287325s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 00:25:21.055213838 (6h59m15.486293969s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 9 00:25:21.326628933 (6h59m15.757709064s) [upload=7311]: Next segment err: metaclient: Too Many Requests Aug 9 00:25:21.326662107 (6h59m15.757742238s) [upload=7311]: Done uploading segments... Aug 9 00:25:21.326682726 (6h59m15.757762907s) [upload=7311]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.492560515s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.267825165s
FAIL (2025-08-09 00:28:54 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 2.267848599s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.271980328s
FAIL (2025-08-09 00:28:33 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 2.271997521s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [X.......X........X..........X....X.....X.X..................] duration:
1.987918322s
FAIL (2025-08-09 00:28:27 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 9 00:15:20.946114856 (6h49m15.377194997s) [upload=7145]: Starting upload
Aug 9 00:15:20.946168780 (6h49m15.377248911s) [upload=7145]: Done starting upload
Aug 9 00:15:20.946228393 (6h49m15.377308525s) [upload=7145]: Uploading segments...
Aug 9 00:15:20.946267749 (6h49m15.37734788s) [upload=7145]: Got next segment. Inline: true
Aug 9 00:15:20.946271877 (6h49m15.377352008s) [upload=7145]: Waiting for error group managing segments...
Aug 9 00:15:20.946306713 (6h49m15.377386845s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 00:15:20.946310951 (6h49m15.377391082s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 9 00:15:20.946313546 (6h49m15.377393677s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 00:15:21.403671645 (6h49m15.834751776s) [upload=7145]: Done uploading segments...
Aug 9 00:15:21.403691763 (6h49m15.834771894s) [upload=7145]: Upload finished. err: <nil>
Duration: 1.987944792s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [................X.......................................X...] duration:
10.793063894s
FAIL (2025-08-09 00:28:22 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:25:21.054812319 (6h59m15.48589246s) [upload=7311]: Starting upload Aug 9 00:25:21.054887373 (6h59m15.485967514s) [upload=7311]: Done starting upload Aug 9 00:25:21.054974740 (6h59m15.486054871s) [upload=7311]: Uploading segments... Aug 9 00:25:21.055143473 (6h59m15.486223604s) [upload=7311]: Got next segment. Inline: false Aug 9 00:25:21.055207194 (6h59m15.486287325s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 00:25:21.055213838 (6h59m15.486293969s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 9 00:25:21.326628933 (6h59m15.757709064s) [upload=7311]: Next segment err: metaclient: Too Many Requests Aug 9 00:25:21.326662107 (6h59m15.757742238s) [upload=7311]: Done uploading segments... Aug 9 00:25:21.326682726 (6h59m15.757762907s) [upload=7311]: Upload finished. err: metaclient: Too Many Requests
Duration: 10.793089693s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
749.301901ms
FAIL (2025-08-09 00:28:04 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 749.324403ms
edge
authservice
http
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.391501815s
FAIL (2025-08-09 00:27:56 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 2.391518748s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................X.......................................X..] duration:
9.915027934s
FAIL (2025-08-09 00:27:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:25:21.054812319 (6h59m15.48589246s) [upload=7311]: Starting upload Aug 9 00:25:21.054887373 (6h59m15.485967514s) [upload=7311]: Done starting upload Aug 9 00:25:21.054974740 (6h59m15.486054871s) [upload=7311]: Uploading segments... Aug 9 00:25:21.055143473 (6h59m15.486223604s) [upload=7311]: Got next segment. Inline: false Aug 9 00:25:21.055207194 (6h59m15.486287325s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 00:25:21.055213838 (6h59m15.486293969s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 9 00:25:21.326628933 (6h59m15.757709064s) [upload=7311]: Next segment err: metaclient: Too Many Requests Aug 9 00:25:21.326662107 (6h59m15.757742238s) [upload=7311]: Done uploading segments... Aug 9 00:25:21.326682726 (6h59m15.757762907s) [upload=7311]: Upload finished. err: metaclient: Too Many Requests
Duration: 9.915054904s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [.X.......X........X..........X....X.....X.X.................] duration:
2.136563102s
FAIL (2025-08-09 00:27:21 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 9 00:15:20.946114856 (6h49m15.377194997s) [upload=7145]: Starting upload
Aug 9 00:15:20.946168780 (6h49m15.377248911s) [upload=7145]: Done starting upload
Aug 9 00:15:20.946228393 (6h49m15.377308525s) [upload=7145]: Uploading segments...
Aug 9 00:15:20.946267749 (6h49m15.37734788s) [upload=7145]: Got next segment. Inline: true
Aug 9 00:15:20.946271877 (6h49m15.377352008s) [upload=7145]: Waiting for error group managing segments...
Aug 9 00:15:20.946306713 (6h49m15.377386845s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 00:15:20.946310951 (6h49m15.377391082s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 9 00:15:20.946313546 (6h49m15.377393677s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 00:15:21.403671645 (6h49m15.834751776s) [upload=7145]: Done uploading segments...
Aug 9 00:15:21.403691763 (6h49m15.834771894s) [upload=7145]: Upload finished. err: <nil>
Duration: 2.136586848s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
770.19221ms
FAIL (2025-08-09 00:27:03 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 770.217759ms
edge
authservice
http
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.460705169s
FAIL (2025-08-09 00:27:02 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 2.460728907s
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
731.11313ms
FAIL (2025-08-09 00:26:46 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 731.132617ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [..X.......X........X..........X....X.....X.X................] duration:
1.903641007s
FAIL (2025-08-09 00:26:45 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 9 00:15:20.946114856 (6h49m15.377194997s) [upload=7145]: Starting upload
Aug 9 00:15:20.946168780 (6h49m15.377248911s) [upload=7145]: Done starting upload
Aug 9 00:15:20.946228393 (6h49m15.377308525s) [upload=7145]: Uploading segments...
Aug 9 00:15:20.946267749 (6h49m15.37734788s) [upload=7145]: Got next segment. Inline: true
Aug 9 00:15:20.946271877 (6h49m15.377352008s) [upload=7145]: Waiting for error group managing segments...
Aug 9 00:15:20.946306713 (6h49m15.377386845s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 00:15:20.946310951 (6h49m15.377391082s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 9 00:15:20.946313546 (6h49m15.377393677s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 00:15:21.403671645 (6h49m15.834751776s) [upload=7145]: Done uploading segments...
Aug 9 00:15:21.403691763 (6h49m15.834771894s) [upload=7145]: Upload finished. err: <nil>
Duration: 1.903674702s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................X.......................................X.] duration:
7.236520346s
FAIL (2025-08-09 00:26:41 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:25:21.054812319 (6h59m15.48589246s) [upload=7311]: Starting upload Aug 9 00:25:21.054887373 (6h59m15.485967514s) [upload=7311]: Done starting upload Aug 9 00:25:21.054974740 (6h59m15.486054871s) [upload=7311]: Uploading segments... Aug 9 00:25:21.055143473 (6h59m15.486223604s) [upload=7311]: Got next segment. Inline: false Aug 9 00:25:21.055207194 (6h59m15.486287325s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 00:25:21.055213838 (6h59m15.486293969s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 9 00:25:21.326628933 (6h59m15.757709064s) [upload=7311]: Next segment err: metaclient: Too Many Requests Aug 9 00:25:21.326662107 (6h59m15.757742238s) [upload=7311]: Done uploading segments... Aug 9 00:25:21.326682726 (6h59m15.757762907s) [upload=7311]: Upload finished. err: metaclient: Too Many Requests
Duration: 7.236546425s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
1.878646765s
FAIL (2025-08-09 00:26:20 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 1.878666383s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [...X.......X........X..........X....X.....X.X...............] duration:
1.903860215s
FAIL (2025-08-09 00:25:58 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 9 00:15:20.946114856 (6h49m15.377194997s) [upload=7145]: Starting upload
Aug 9 00:15:20.946168780 (6h49m15.377248911s) [upload=7145]: Done starting upload
Aug 9 00:15:20.946228393 (6h49m15.377308525s) [upload=7145]: Uploading segments...
Aug 9 00:15:20.946267749 (6h49m15.37734788s) [upload=7145]: Got next segment. Inline: true
Aug 9 00:15:20.946271877 (6h49m15.377352008s) [upload=7145]: Waiting for error group managing segments...
Aug 9 00:15:20.946306713 (6h49m15.377386845s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 00:15:20.946310951 (6h49m15.377391082s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 9 00:15:20.946313546 (6h49m15.377393677s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 00:15:21.403671645 (6h49m15.834751776s) [upload=7145]: Done uploading segments...
Aug 9 00:15:21.403691763 (6h49m15.834771894s) [upload=7145]: Upload finished. err: <nil>
Duration: 1.903889582s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.522706546s
FAIL (2025-08-09 00:25:42 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 2.522732356s
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
718.062982ms
FAIL (2025-08-09 00:25:42 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 718.089833ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................X.......................................X] duration:
923.045874ms
FAIL (2025-08-09 00:25:21 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 9 00:25:21.054812319 (6h59m15.48589246s) [upload=7311]: Starting upload Aug 9 00:25:21.054887373 (6h59m15.485967514s) [upload=7311]: Done starting upload Aug 9 00:25:21.054974740 (6h59m15.486054871s) [upload=7311]: Uploading segments... Aug 9 00:25:21.055143473 (6h59m15.486223604s) [upload=7311]: Got next segment. Inline: false Aug 9 00:25:21.055207194 (6h59m15.486287325s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 9 00:25:21.055213838 (6h59m15.486293969s) [upload=7311]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 9 00:25:21.326628933 (6h59m15.757709064s) [upload=7311]: Next segment err: metaclient: Too Many Requests Aug 9 00:25:21.326662107 (6h59m15.757742238s) [upload=7311]: Done uploading segments... Aug 9 00:25:21.326682726 (6h59m15.757762907s) [upload=7311]: Upload finished. err: metaclient: Too Many Requests
Duration: 923.060712ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
742.998822ms
FAIL (2025-08-09 00:25:17 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 743.030081ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [....X.......X........X..........X....X.....X.X..............] duration:
1.873129258s
FAIL (2025-08-09 00:24:58 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 9 00:15:20.946114856 (6h49m15.377194997s) [upload=7145]: Starting upload
Aug 9 00:15:20.946168780 (6h49m15.377248911s) [upload=7145]: Done starting upload
Aug 9 00:15:20.946228393 (6h49m15.377308525s) [upload=7145]: Uploading segments...
Aug 9 00:15:20.946267749 (6h49m15.37734788s) [upload=7145]: Got next segment. Inline: true
Aug 9 00:15:20.946271877 (6h49m15.377352008s) [upload=7145]: Waiting for error group managing segments...
Aug 9 00:15:20.946306713 (6h49m15.377386845s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 00:15:20.946310951 (6h49m15.377391082s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 9 00:15:20.946313546 (6h49m15.377393677s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 00:15:21.403671645 (6h49m15.834751776s) [upload=7145]: Done uploading segments...
Aug 9 00:15:21.403691763 (6h49m15.834771894s) [upload=7145]: Upload finished. err: <nil>
Duration: 1.873156349s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
753.220518ms
FAIL (2025-08-09 00:24:49 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 753.245957ms
edge
authservice
http
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.582000515s
FAIL (2025-08-09 00:24:47 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 2.582026393s
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
746.199185ms
FAIL (2025-08-09 00:24:34 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 746.227749ms
edge
authservice
http
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.502280576s
FAIL (2025-08-09 00:24:26 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded
Duration: 2.502304803s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
6.429043489s
FAIL (2025-08-09 00:24:23 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b8d209ce06ccb9d43dcc65ea2f36c72b0d150208b68a0946d6edbdd15267451f != before 5d5ee590aac95b51ee6922a1c4bbaa5058c66548965c5c82421b5579e073ad46
======LOGS======
Aug 8 23:52:54.319517211 (6h26m48.750597342s) [upload=6764]: Starting upload
Aug 8 23:52:54.319570443 (6h26m48.750650584s) [upload=6764]: Done starting upload
Aug 8 23:52:54.319637891 (6h26m48.750718022s) [upload=6764]: Uploading segments...
Aug 8 23:52:54.319770525 (6h26m48.750850656s) [upload=6764]: Got next segment. Inline: false
Aug 8 23:52:54.319800643 (6h26m48.750880774s) [upload=6764]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 23:52:54.319806814 (6h26m48.750886945s) [upload=6764]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 23:52:54.761193680 (6h26m49.192273841s) [seg_pos={0 0}, upload=6764]: Begin upload segment...
Aug 8 23:52:54.761237794 (6h26m49.192317925s) [seg_pos={0 0}, upload=6764]: Done begin upload segment.
Aug 8 23:52:54.761257642 (6h26m49.192337773s) [noise=true , address=104.36.82.11:4117 , piece_id=N4N7DCNMGIFHSCNHQST3JHC3SW555XZEPSA6NGS7WNS7XGGASU6Q, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.761327066 (6h26m49.192407197s) [noise=true , address=104.36.82.59:4318 , piece_id=JVKQ6XZXKBLVMPHSQPHVNFCDMHSQ6GHDCLZFMZPV7AGJGYI2ZDZQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.761687635 (6h26m49.192767756s) [noise=true , address=64.49.13.154:4927 , piece_id=OJYSXPOVBO65AWFYUZDWZK7QGJ6J4ZR46OKYSDECRB36D4N4KR6A, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.761744604 (6h26m49.192824735s) [noise=true , address=104.36.82.23:4462 , piece_id=NCRSXLXGW7WTN7C5VPEL2RBBYHC6NSPORND7QLXNHK2DGB4P67FQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.761997258 (6h26m49.193077379s) [noise=true , address=104.36.82.19:4408 , piece_id=X5EYFVZ7M2KUDIX3VR3RIZ7JL3QLZDBJBUE7LY7AB3OMSWCCSTCQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.762040070 (6h26m49.193120201s) [noise=true , address=45.91.116.9:4287 , piece_id=WCLJWCMCKD7Q2IIAPGJMPZ3WLVFRKDRKTHMQBRPAQ5N3FL3KCFCA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.762306589 (6h26m49.19338671s) [noise=true , address=104.36.82.58:4287 , piece_id=FE2LXTAY7ILMLJJIA3GT766MQJYJ42EJRU6GXQTCJQEMU46B4AVA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.762315898 (6h26m49.193396029s) [noise=true , address=104.36.82.36:4630 , piece_id=XYYBSDSS3T556YC6X3GVWEI3ZJNLC52TM5EA5TGEUU5NPPQPLGFA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.762600652 (6h26m49.193680783s) [noise=true , address=193.149.164.143:4695 , piece_id=IFE7KQUCEN2LPEUOIAC4HGLBKWRKMKEPU6JEJKHTAJLYUWTBYMPA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.762762913 (6h26m49.193843044s) [noise=true , address=104.36.82.43:4866 , piece_id=PBOZ6O2NABE2IMFJKFKGURVLVVVRDFTKEMRCEMUUL5XACR7Q2ILQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.762890787 (6h26m49.193970918s) [noise=true , address=45.91.116.8:4071 , piece_id=WV2PW4KOQRYAJA7LK2SOQYIPUMO6URXCQMN66DW6QL6CHTTALZ4A, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763084147 (6h26m49.194164268s) [noise=true , address=104.36.82.8:4993 , piece_id=N27LRE6SHS4NHEBFNKWRC6HB3OD6FKD2OUE766U4M2VDUSZBPSNA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763201191 (6h26m49.194281322s) [noise=true , address=104.36.82.22:4543 , piece_id=6HLZOT2NTFGAGYBPDK3BSZ64CUFK7ORY3C7JTSGJTQ7HI53JOPDQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763333274 (6h26m49.194413405s) [noise=true , address=104.36.82.60:4359 , piece_id=SQSHETBRMMSPFRASJEHJTPWOXUKKQ55WTY3G4HDJF6WBABT5H7NA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763430680 (6h26m49.194510811s) [noise=true , address=79.127.205.233:4633 , piece_id=VSOM6JFGGHLYA5AUGHMLBNNRIOBJM6OV6DF2JYHNOGROVBWA4HPQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763564055 (6h26m49.194644186s) [noise=true , address=104.36.82.37:4681 , piece_id=EYPY2U7WMUAJY7LUYWV4MIJT3AKIB536Y4HNDCXDXPUYRPTVLAGA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763770510 (6h26m49.194850641s) [noise=true , address=104.36.82.36:4641 , piece_id=AABZ6AD6VW2U7HLN3GSHLVEVBILRQRA64ZMCQBNOV3BHV77ECEQQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763971715 (6h26m49.195051847s) [noise=true , address=104.36.82.16:4303 , piece_id=7MNEWHK2325OMRFZQGLXTSUB6VQPWETGE2RB5Y5CS6DNQLUAZP5A, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:55.862989103 (6h26m50.294069224s) [upload=6764]: Next returned nil segment
Aug 8 23:52:55.863005615 (6h26m50.294085746s) [upload=6764]: Waiting for error group managing segments...
Aug 8 23:52:56.141209853 (6h26m50.572289984s) [noise=true , address=79.127.205.233:4633 , piece_id=VSOM6JFGGHLYA5AUGHMLBNNRIOBJM6OV6DF2JYHNOGROVBWA4HPQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.198220311 (6h26m50.629300442s) [noise=true , address=64.49.13.154:4927 , piece_id=OJYSXPOVBO65AWFYUZDWZK7QGJ6J4ZR46OKYSDECRB36D4N4KR6A, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.198927263 (6h26m50.630007395s) [noise=true , address=193.149.164.143:4695 , piece_id=IFE7KQUCEN2LPEUOIAC4HGLBKWRKMKEPU6JEJKHTAJLYUWTBYMPA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.672532932 (6h26m51.103613083s) [noise=true , address=104.36.82.43:4866 , piece_id=PBOZ6O2NABE2IMFJKFKGURVLVVVRDFTKEMRCEMUUL5XACR7Q2ILQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.695171757 (6h26m51.126251888s) [noise=true , address=104.36.82.22:4543 , piece_id=6HLZOT2NTFGAGYBPDK3BSZ64CUFK7ORY3C7JTSGJTQ7HI53JOPDQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.710751747 (6h26m51.141831878s) [noise=true , address=104.36.82.59:4318 , piece_id=JVKQ6XZXKBLVMPHSQPHVNFCDMHSQ6GHDCLZFMZPV7AGJGYI2ZDZQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.735069323 (6h26m51.166149444s) [noise=true , address=45.91.116.9:4287 , piece_id=WCLJWCMCKD7Q2IIAPGJMPZ3WLVFRKDRKTHMQBRPAQ5N3FL3KCFCA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.767179780 (6h26m51.198259911s) [noise=true , address=104.36.82.36:4630 , piece_id=XYYBSDSS3T556YC6X3GVWEI3ZJNLC52TM5EA5TGEUU5NPPQPLGFA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.830162715 (6h26m51.261242846s) [noise=true , address=104.36.82.19:4408 , piece_id=X5EYFVZ7M2KUDIX3VR3RIZ7JL3QLZDBJBUE7LY7AB3OMSWCCSTCQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.860014180 (6h26m51.291094311s) [noise=true , address=104.36.82.37:4681 , piece_id=EYPY2U7WMUAJY7LUYWV4MIJT3AKIB536Y4HNDCXDXPUYRPTVLAGA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.872003299 (6h26m51.30308343s) [noise=true , address=104.36.82.11:4117 , piece_id=N4N7DCNMGIFHSCNHQST3JHC3SW555XZEPSA6NGS7WNS7XGGASU6Q, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.877331624 (6h26m51.308411755s) [noise=true , address=104.36.82.36:4641 , piece_id=AABZ6AD6VW2U7HLN3GSHLVEVBILRQRA64ZMCQBNOV3BHV77ECEQQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.893061600 (6h26m51.324141731s) [noise=true , address=104.36.82.16:4303 , piece_id=7MNEWHK2325OMRFZQGLXTSUB6VQPWETGE2RB5Y5CS6DNQLUAZP5A, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.978339422 (6h26m51.409419563s) [noise=true , address=45.91.116.8:4071 , piece_id=WV2PW4KOQRYAJA7LK2SOQYIPUMO6URXCQMN66DW6QL6CHTTALZ4A, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.978373277 (6h26m51.409453408s) [seg_pos={0 0}, upload=6764]: Segment reached optimal threshold of 14 pieces.
Aug 8 23:52:56.978498937 (6h26m51.409579068s) [noise=true , address=104.36.82.8:4993 , piece_id=N27LRE6SHS4NHEBFNKWRC6HB3OD6FKD2OUE766U4M2VDUSZBPSNA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 8 23:52:56.978716593 (6h26m51.409796715s) [noise=true , address=104.36.82.58:4287 , piece_id=FE2LXTAY7ILMLJJIA3GT766MQJYJ42EJRU6GXQTCJQEMU46B4AVA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wNKharWFFSSsLMd1vZYi6zYEYBe3VT6Z3ikrY7Cjo3xPLUK62): protocol: context canceled; context canceled
Aug 8 23:52:56.978783031 (6h26m51.409863152s) [noise=true , address=104.36.82.60:4359 , piece_id=SQSHETBRMMSPFRASJEHJTPWOXUKKQ55WTY3G4HDJF6WBABT5H7NA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YgFc7P9RCbYw9u2wfow8Z3DMwFvJ6AnCsvzoxU9pTwPWGw8MV): protocol: rpc: dial tcp 104.36.82.60:4359: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 23:52:56.978860258 (6h26m51.409940389s) [noise=true , address=104.36.82.23:4462 , piece_id=NCRSXLXGW7WTN7C5VPEL2RBBYHC6NSPORND7QLXNHK2DGB4P67FQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu): protocol: rpc: dial tcp 104.36.82.23:4462: operation was canceled; context canceled
Aug 8 23:52:56.978883112 (6h26m51.409963243s) [seg_pos={0 0}, upload=6764]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 23:52:56.978961131 (6h26m51.410041263s) [upload=6764]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 23:52:56.978965379 (6h26m51.4100455s) [upload=6764]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 23:52:57.271502311 (6h26m51.702582452s) [upload=6764]: Done uploading segments...
Aug 8 23:52:57.271522960 (6h26m51.702603091s) [upload=6764]: Upload finished. err: <nil>
Duration: 6.429066833s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [.....X.......X........X..........X....X.....X.X.............] duration:
2.106326381s
FAIL (2025-08-09 00:24:19 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 9 00:15:20.946114856 (6h49m15.377194997s) [upload=7145]: Starting upload
Aug 9 00:15:20.946168780 (6h49m15.377248911s) [upload=7145]: Done starting upload
Aug 9 00:15:20.946228393 (6h49m15.377308525s) [upload=7145]: Uploading segments...
Aug 9 00:15:20.946267749 (6h49m15.37734788s) [upload=7145]: Got next segment. Inline: true
Aug 9 00:15:20.946271877 (6h49m15.377352008s) [upload=7145]: Waiting for error group managing segments...
Aug 9 00:15:20.946306713 (6h49m15.377386845s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 00:15:20.946310951 (6h49m15.377391082s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 9 00:15:20.946313546 (6h49m15.377393677s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 00:15:21.403671645 (6h49m15.834751776s) [upload=7145]: Done uploading segments...
Aug 9 00:15:21.403691763 (6h49m15.834771894s) [upload=7145]: Upload finished. err: <nil>
Duration: 2.10635196s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
9.133483664s
FAIL (2025-08-09 00:24:05 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after b8d209ce06ccb9d43dcc65ea2f36c72b0d150208b68a0946d6edbdd15267451f != before 5d5ee590aac95b51ee6922a1c4bbaa5058c66548965c5c82421b5579e073ad46
======LOGS======
Aug 8 23:52:54.319517211 (6h26m48.750597342s) [upload=6764]: Starting upload
Aug 8 23:52:54.319570443 (6h26m48.750650584s) [upload=6764]: Done starting upload
Aug 8 23:52:54.319637891 (6h26m48.750718022s) [upload=6764]: Uploading segments...
Aug 8 23:52:54.319770525 (6h26m48.750850656s) [upload=6764]: Got next segment. Inline: false
Aug 8 23:52:54.319800643 (6h26m48.750880774s) [upload=6764]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 23:52:54.319806814 (6h26m48.750886945s) [upload=6764]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 23:52:54.761193680 (6h26m49.192273841s) [seg_pos={0 0}, upload=6764]: Begin upload segment...
Aug 8 23:52:54.761237794 (6h26m49.192317925s) [seg_pos={0 0}, upload=6764]: Done begin upload segment.
Aug 8 23:52:54.761257642 (6h26m49.192337773s) [noise=true , address=104.36.82.11:4117 , piece_id=N4N7DCNMGIFHSCNHQST3JHC3SW555XZEPSA6NGS7WNS7XGGASU6Q, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.761327066 (6h26m49.192407197s) [noise=true , address=104.36.82.59:4318 , piece_id=JVKQ6XZXKBLVMPHSQPHVNFCDMHSQ6GHDCLZFMZPV7AGJGYI2ZDZQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.761687635 (6h26m49.192767756s) [noise=true , address=64.49.13.154:4927 , piece_id=OJYSXPOVBO65AWFYUZDWZK7QGJ6J4ZR46OKYSDECRB36D4N4KR6A, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.761744604 (6h26m49.192824735s) [noise=true , address=104.36.82.23:4462 , piece_id=NCRSXLXGW7WTN7C5VPEL2RBBYHC6NSPORND7QLXNHK2DGB4P67FQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.761997258 (6h26m49.193077379s) [noise=true , address=104.36.82.19:4408 , piece_id=X5EYFVZ7M2KUDIX3VR3RIZ7JL3QLZDBJBUE7LY7AB3OMSWCCSTCQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.762040070 (6h26m49.193120201s) [noise=true , address=45.91.116.9:4287 , piece_id=WCLJWCMCKD7Q2IIAPGJMPZ3WLVFRKDRKTHMQBRPAQ5N3FL3KCFCA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.762306589 (6h26m49.19338671s) [noise=true , address=104.36.82.58:4287 , piece_id=FE2LXTAY7ILMLJJIA3GT766MQJYJ42EJRU6GXQTCJQEMU46B4AVA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.762315898 (6h26m49.193396029s) [noise=true , address=104.36.82.36:4630 , piece_id=XYYBSDSS3T556YC6X3GVWEI3ZJNLC52TM5EA5TGEUU5NPPQPLGFA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.762600652 (6h26m49.193680783s) [noise=true , address=193.149.164.143:4695 , piece_id=IFE7KQUCEN2LPEUOIAC4HGLBKWRKMKEPU6JEJKHTAJLYUWTBYMPA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.762762913 (6h26m49.193843044s) [noise=true , address=104.36.82.43:4866 , piece_id=PBOZ6O2NABE2IMFJKFKGURVLVVVRDFTKEMRCEMUUL5XACR7Q2ILQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.762890787 (6h26m49.193970918s) [noise=true , address=45.91.116.8:4071 , piece_id=WV2PW4KOQRYAJA7LK2SOQYIPUMO6URXCQMN66DW6QL6CHTTALZ4A, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763084147 (6h26m49.194164268s) [noise=true , address=104.36.82.8:4993 , piece_id=N27LRE6SHS4NHEBFNKWRC6HB3OD6FKD2OUE766U4M2VDUSZBPSNA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763201191 (6h26m49.194281322s) [noise=true , address=104.36.82.22:4543 , piece_id=6HLZOT2NTFGAGYBPDK3BSZ64CUFK7ORY3C7JTSGJTQ7HI53JOPDQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763333274 (6h26m49.194413405s) [noise=true , address=104.36.82.60:4359 , piece_id=SQSHETBRMMSPFRASJEHJTPWOXUKKQ55WTY3G4HDJF6WBABT5H7NA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763430680 (6h26m49.194510811s) [noise=true , address=79.127.205.233:4633 , piece_id=VSOM6JFGGHLYA5AUGHMLBNNRIOBJM6OV6DF2JYHNOGROVBWA4HPQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763564055 (6h26m49.194644186s) [noise=true , address=104.36.82.37:4681 , piece_id=EYPY2U7WMUAJY7LUYWV4MIJT3AKIB536Y4HNDCXDXPUYRPTVLAGA, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763770510 (6h26m49.194850641s) [noise=true , address=104.36.82.36:4641 , piece_id=AABZ6AD6VW2U7HLN3GSHLVEVBILRQRA64ZMCQBNOV3BHV77ECEQQ, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:54.763971715 (6h26m49.195051847s) [noise=true , address=104.36.82.16:4303 , piece_id=7MNEWHK2325OMRFZQGLXTSUB6VQPWETGE2RB5Y5CS6DNQLUAZP5A, seg_pos={0 0}, upload=6764]: Uploading piece...
Aug 8 23:52:55.862989103 (6h26m50.294069224s) [upload=6764]: Next returned nil segment
Aug 8 23:52:55.863005615 (6h26m50.294085746s) [upload=6764]: Waiting for error group managing segments...
Aug 8 23:52:56.141209853 (6h26m50.572289984s) [noise=true , address=79.127.205.233:4633 , piece_id=VSOM6JFGGHLYA5AUGHMLBNNRIOBJM6OV6DF2JYHNOGROVBWA4HPQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.198220311 (6h26m50.629300442s) [noise=true , address=64.49.13.154:4927 , piece_id=OJYSXPOVBO65AWFYUZDWZK7QGJ6J4ZR46OKYSDECRB36D4N4KR6A, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.198927263 (6h26m50.630007395s) [noise=true , address=193.149.164.143:4695 , piece_id=IFE7KQUCEN2LPEUOIAC4HGLBKWRKMKEPU6JEJKHTAJLYUWTBYMPA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.672532932 (6h26m51.103613083s) [noise=true , address=104.36.82.43:4866 , piece_id=PBOZ6O2NABE2IMFJKFKGURVLVVVRDFTKEMRCEMUUL5XACR7Q2ILQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.695171757 (6h26m51.126251888s) [noise=true , address=104.36.82.22:4543 , piece_id=6HLZOT2NTFGAGYBPDK3BSZ64CUFK7ORY3C7JTSGJTQ7HI53JOPDQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.710751747 (6h26m51.141831878s) [noise=true , address=104.36.82.59:4318 , piece_id=JVKQ6XZXKBLVMPHSQPHVNFCDMHSQ6GHDCLZFMZPV7AGJGYI2ZDZQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.735069323 (6h26m51.166149444s) [noise=true , address=45.91.116.9:4287 , piece_id=WCLJWCMCKD7Q2IIAPGJMPZ3WLVFRKDRKTHMQBRPAQ5N3FL3KCFCA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.767179780 (6h26m51.198259911s) [noise=true , address=104.36.82.36:4630 , piece_id=XYYBSDSS3T556YC6X3GVWEI3ZJNLC52TM5EA5TGEUU5NPPQPLGFA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.830162715 (6h26m51.261242846s) [noise=true , address=104.36.82.19:4408 , piece_id=X5EYFVZ7M2KUDIX3VR3RIZ7JL3QLZDBJBUE7LY7AB3OMSWCCSTCQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.860014180 (6h26m51.291094311s) [noise=true , address=104.36.82.37:4681 , piece_id=EYPY2U7WMUAJY7LUYWV4MIJT3AKIB536Y4HNDCXDXPUYRPTVLAGA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.872003299 (6h26m51.30308343s) [noise=true , address=104.36.82.11:4117 , piece_id=N4N7DCNMGIFHSCNHQST3JHC3SW555XZEPSA6NGS7WNS7XGGASU6Q, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.877331624 (6h26m51.308411755s) [noise=true , address=104.36.82.36:4641 , piece_id=AABZ6AD6VW2U7HLN3GSHLVEVBILRQRA64ZMCQBNOV3BHV77ECEQQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.893061600 (6h26m51.324141731s) [noise=true , address=104.36.82.16:4303 , piece_id=7MNEWHK2325OMRFZQGLXTSUB6VQPWETGE2RB5Y5CS6DNQLUAZP5A, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.978339422 (6h26m51.409419563s) [noise=true , address=45.91.116.8:4071 , piece_id=WV2PW4KOQRYAJA7LK2SOQYIPUMO6URXCQMN66DW6QL6CHTTALZ4A, seg_pos={0 0}, upload=6764]: Done uploading piece. err: <nil>
Aug 8 23:52:56.978373277 (6h26m51.409453408s) [seg_pos={0 0}, upload=6764]: Segment reached optimal threshold of 14 pieces.
Aug 8 23:52:56.978498937 (6h26m51.409579068s) [noise=true , address=104.36.82.8:4993 , piece_id=N27LRE6SHS4NHEBFNKWRC6HB3OD6FKD2OUE766U4M2VDUSZBPSNA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xnx9shwwZAcWVV6aPj2ikukj8nbKiFsBs1hiS89zSkRQqNMJD): protocol: expected piece hash; context canceled; EOF
Aug 8 23:52:56.978716593 (6h26m51.409796715s) [noise=true , address=104.36.82.58:4287 , piece_id=FE2LXTAY7ILMLJJIA3GT766MQJYJ42EJRU6GXQTCJQEMU46B4AVA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wNKharWFFSSsLMd1vZYi6zYEYBe3VT6Z3ikrY7Cjo3xPLUK62): protocol: context canceled; context canceled
Aug 8 23:52:56.978783031 (6h26m51.409863152s) [noise=true , address=104.36.82.60:4359 , piece_id=SQSHETBRMMSPFRASJEHJTPWOXUKKQ55WTY3G4HDJF6WBABT5H7NA, seg_pos={0 0}, upload=6764]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YgFc7P9RCbYw9u2wfow8Z3DMwFvJ6AnCsvzoxU9pTwPWGw8MV): protocol: rpc: dial tcp 104.36.82.60:4359: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 23:52:56.978860258 (6h26m51.409940389s) [noise=true , address=104.36.82.23:4462 , piece_id=NCRSXLXGW7WTN7C5VPEL2RBBYHC6NSPORND7QLXNHK2DGB4P67FQ, seg_pos={0 0}, upload=6764]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YirC8WVSoE1qrWGMmayNmCsDLxjUhggrgXL5unKtQjsfUkbMu): protocol: rpc: dial tcp 104.36.82.23:4462: operation was canceled; context canceled
Aug 8 23:52:56.978883112 (6h26m51.409963243s) [seg_pos={0 0}, upload=6764]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 23:52:56.978961131 (6h26m51.410041263s) [upload=6764]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 23:52:56.978965379 (6h26m51.4100455s) [upload=6764]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 23:52:57.271502311 (6h26m51.702582452s) [upload=6764]: Done uploading segments...
Aug 8 23:52:57.271522960 (6h26m51.702603091s) [upload=6764]: Upload finished. err: <nil>
Duration: 9.13351254s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
716.330698ms
FAIL (2025-08-09 00:23:30 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": dial tcp 45.38.24.2:443: i/o timeout
Duration: 716.360234ms
edge
authservice
http
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
7 out of 60 is failed (threshold=3) [......X.......X........X..........X....X.....X.X............] duration:
1.95839214s
FAIL (2025-08-09 00:23:21 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: object not found ("orbiter_us_0/1K")
======LOGS======
Aug 9 00:15:20.946114856 (6h49m15.377194997s) [upload=7145]: Starting upload
Aug 9 00:15:20.946168780 (6h49m15.377248911s) [upload=7145]: Done starting upload
Aug 9 00:15:20.946228393 (6h49m15.377308525s) [upload=7145]: Uploading segments...
Aug 9 00:15:20.946267749 (6h49m15.37734788s) [upload=7145]: Got next segment. Inline: true
Aug 9 00:15:20.946271877 (6h49m15.377352008s) [upload=7145]: Waiting for error group managing segments...
Aug 9 00:15:20.946306713 (6h49m15.377386845s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 9 00:15:20.946310951 (6h49m15.377391082s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 9 00:15:20.946313546 (6h49m15.377393677s) [upload=7145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 9 00:15:21.403671645 (6h49m15.834751776s) [upload=7145]: Done uploading segments...
Aug 9 00:15:21.403691763 (6h49m15.834771894s) [upload=7145]: Upload finished. err: <nil>
Duration: 1.958417779s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|