Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..........X......................X...........X............X.] duration:
5.714418074s trace:
https://jaeger.dev.storj.io/trace/40fe15a1a44c00b0
FAIL (2026-05-13 19:53:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.714429666s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...........X......................X...........X............X] duration:
1m0.000410706s trace:
https://jaeger.dev.storj.io/trace/40fe15a1a44c00b0
FAIL (2026-05-13 19:52:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000435033s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [............X......................X...........X............] duration:
6.840814801s trace:
https://jaeger.dev.storj.io/trace/6f0858f594da7482
FAIL (2026-05-13 19:51:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.84084045s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [X............X......................X...........X...........] duration:
6.858107076s trace:
https://jaeger.dev.storj.io/trace/6f0858f594da7482
FAIL (2026-05-13 19:50:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.858139639s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.X............X......................X...........X..........] duration:
27.502239728s trace:
https://jaeger.dev.storj.io/trace/6f0858f594da7482
FAIL (2026-05-13 19:50:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 27.502256501s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..X............X......................X...........X.........] duration:
7.86995753s trace:
https://jaeger.dev.storj.io/trace/6f0858f594da7482
FAIL (2026-05-13 19:49:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.869982096s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...X............X......................X...........X........] duration:
6.783647599s trace:
https://jaeger.dev.storj.io/trace/6f0858f594da7482
FAIL (2026-05-13 19:48:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.783665492s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [X...X............X......................X...........X.......] duration:
6.875173706s trace:
https://jaeger.dev.storj.io/trace/6f0858f594da7482
FAIL (2026-05-13 19:47:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.87524331s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.X...X............X......................X...........X......] duration:
6.720461018s trace:
https://jaeger.dev.storj.io/trace/6f0858f594da7482
FAIL (2026-05-13 19:47:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.720662002s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [..X...X............X......................X...........X.....] duration:
6.627088894s trace:
https://jaeger.dev.storj.io/trace/6f0858f594da7482
FAIL (2026-05-13 19:46:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.627114132s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [...X...X............X......................X...........X....] duration:
6.698155188s trace:
https://jaeger.dev.storj.io/trace/6f0858f594da7482
FAIL (2026-05-13 19:45:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.698169976s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [....X...X............X......................X...........X...] duration:
6.837077115s trace:
https://jaeger.dev.storj.io/trace/6f0858f594da7482
FAIL (2026-05-13 19:44:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.837099948s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.....X...X............X......................X...........X..] duration:
6.531788266s trace:
https://jaeger.dev.storj.io/trace/6f0858f594da7482
FAIL (2026-05-13 19:43:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.531798978s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [......X...X............X......................X...........X.] duration:
6.613773539s trace:
https://jaeger.dev.storj.io/trace/6f0858f594da7482
FAIL (2026-05-13 19:42:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.613788538s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.......X...X............X......................X...........X] duration:
1m0.00110198s trace:
https://jaeger.dev.storj.io/trace/6f0858f594da7482
FAIL (2026-05-13 19:42:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001123402s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [........X...X............X......................X...........] duration:
6.851149584s trace:
https://jaeger.dev.storj.io/trace/2ab951cfb145aac9
FAIL (2026-05-13 19:40:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.85117375s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.........X...X............X......................X..........] duration:
6.658775466s trace:
https://jaeger.dev.storj.io/trace/2ab951cfb145aac9
FAIL (2026-05-13 19:40:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.658802919s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..........X...X............X......................X.........] duration:
6.90684393s trace:
https://jaeger.dev.storj.io/trace/2ab951cfb145aac9
FAIL (2026-05-13 19:39:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.906875388s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...........X...X............X......................X........] duration:
6.680157795s trace:
https://jaeger.dev.storj.io/trace/2ab951cfb145aac9
FAIL (2026-05-13 19:38:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.68036986s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [............X...X............X......................X.......] duration:
7.006970487s trace:
https://jaeger.dev.storj.io/trace/2ab951cfb145aac9
FAIL (2026-05-13 19:37:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.006993732s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.............X...X............X......................X......] duration:
6.749132356s trace:
https://jaeger.dev.storj.io/trace/2ab951cfb145aac9
FAIL (2026-05-13 19:37:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.74915555s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..............X...X............X......................X.....] duration:
6.557285965s trace:
https://jaeger.dev.storj.io/trace/2ab951cfb145aac9
FAIL (2026-05-13 19:35:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.55729875s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...............X...X............X......................X....] duration:
6.60614863s trace:
https://jaeger.dev.storj.io/trace/2ab951cfb145aac9
FAIL (2026-05-13 19:34:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.606192843s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [................X...X............X......................X...] duration:
6.636764809s trace:
https://jaeger.dev.storj.io/trace/2ab951cfb145aac9
FAIL (2026-05-13 19:34:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.636787391s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.................X...X............X......................X..] duration:
6.841131882s trace:
https://jaeger.dev.storj.io/trace/2ab951cfb145aac9
FAIL (2026-05-13 19:34:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.841148064s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..................X...X............X......................X.] duration:
17.361181269s trace:
https://jaeger.dev.storj.io/trace/2ab951cfb145aac9
FAIL (2026-05-13 19:33:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 17.361193731s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...................X...X............X......................X] duration:
1m0.000715985s trace:
https://jaeger.dev.storj.io/trace/2ab951cfb145aac9
FAIL (2026-05-13 19:33:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.00073981s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [....................X...X............X......................] duration:
6.802451179s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:31:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.802478641s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.....................X...X............X.....................] duration:
11.505205138s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:30:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 11.505222982s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [......................X...X............X....................] duration:
6.735757876s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:30:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.735785208s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.......................X...X............X...................] duration:
6.798689373s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:29:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.798702899s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [........................X...X............X..................] duration:
6.860293967s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:28:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.860318242s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.........................X...X............X.................] duration:
6.718867016s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:27:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.718888919s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..........................X...X............X................] duration:
6.824549739s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:26:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.824665259s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...........................X...X............X...............] duration:
6.858829951s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:25:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.858858205s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [............................X...X............X..............] duration:
6.588306063s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:24:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.588381085s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.............................X...X............X.............] duration:
7.041532676s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:23:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.041615864s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..............................X...X............X............] duration:
28.171813525s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:22:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 28.172160046s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
6.069035135s trace:
https://jaeger.dev.storj.io/trace/61f95f9c55543be9
FAIL (2026-05-13 19:22:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
May 13 18:42:39.654115046 (311h54m50.174938791s) [upload=298824]: Starting upload
May 13 18:42:39.654237830 (311h54m50.175061595s) [upload=298824]: Done starting upload
May 13 18:42:39.654294318 (311h54m50.175118103s) [upload=298824]: Uploading segments...
May 13 18:42:39.654647201 (311h54m50.175470986s) [upload=298824]: Got next segment. Inline: false
May 13 18:42:39.654688680 (311h54m50.175512425s) [upload=298824]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 18:42:39.654695634 (311h54m50.175519379s) [upload=298824]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 18:42:39.943710770 (311h54m50.464534515s) [seg_pos={0 0}, upload=298824]: Begin upload segment...
May 13 18:42:39.943793989 (311h54m50.464617765s) [seg_pos={0 0}, upload=298824]: Done begin upload segment.
May 13 18:42:39.943823095 (311h54m50.46464685s) [noise=true , address=145.79.227.50:4380 , piece_id=BOO6A7PMYBGEE4PT6QUXUDBEEIE2P6G3BAHZXDKQIW5BEROQE6QA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.943840539 (311h54m50.464664284s) [noise=true , address=193.149.164.71:4294 , piece_id=JNC7X62V4WU7N64M5SMBYCFFJRUSEJUV2F5EET5MEQN3CL7REKWQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944027826 (311h54m50.464851571s) [noise=true , address=77.90.31.17:4791 , piece_id=2AJ2EQNEN2EAEOOO7ZWMF6BZNR5LMPYT7OGK6XQJL6NKF7NQF26Q, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944491761 (311h54m50.465315506s) [noise=true , address=193.149.164.68:4056 , piece_id=5XNXS6DFS7EOWWKYJ5BDVFDUJPGCYQWT4L3HNH5JBMXJDH67WLXQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944901293 (311h54m50.465725038s) [noise=true , address=145.79.227.6:4277 , piece_id=I6VBYHUGW4ENFKM6PWFNDNGVDZCZTUPWSJLTGAW4OXB3GMEUS3EA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.945324029 (311h54m50.466147774s) [noise=true , address=167.17.72.71:4793 , piece_id=SAEMFXJYRFHIJDBZFYAR7AXLDPHAXWTSFSIVKATGZ5SLFKQNOYVQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.945722740 (311h54m50.466546485s) [noise=true , address=45.91.116.9:4279 , piece_id=URBFWN55CEWJ6NHVUB3SABQLY7LM7IQTQUIWW2RFJVDOHNUIJRFA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.946124527 (311h54m50.466948282s) [noise=true , address=45.91.116.14:4220 , piece_id=EDKJQP6N2GL2XKRCKNZV5TOKQK24OCKYYCLVSSBH6YHBLHWAIPNQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.946494653 (311h54m50.467318398s) [noise=true , address=167.17.72.70:4778 , piece_id=7754ZHKMHT7GREAJKTPJ6VUOGTNU243QFPMJNIPYMEJK4QREYD4A, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.947965089 (311h54m50.468788834s) [noise=true , address=77.90.31.14:4691 , piece_id=JFI56VQMCJZRDD2BZ3GOSKFLNVIVHEV6U3WK7MZ6LDU77ZGPOJNQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.948256265 (311h54m50.46908001s) [noise=true , address=77.90.31.15:4722 , piece_id=FHQJJBOHMFGMJ3BKJJJBOPLVATIA2AIC64XPOE25WG6IP6NN55PA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.950216525 (311h54m50.47104027s) [noise=true , address=45.91.116.10:4110 , piece_id=BC5TDQDVKMQFESE5KYRT2BSS62JKEZFX33Y2IXB5S76DGNXF352Q, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.951870151 (311h54m50.472693896s) [noise=true , address=45.91.116.4:4942 , piece_id=STFL2OELEKJASIEZFAJ7HVRTCKKVTKY5LK5S5CYSRXZUZLQEOQDQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.953453131 (311h54m50.474276876s) [noise=true , address=145.79.227.47:4282 , piece_id=UZ4CAH5KR2XUABI6KFT4QIE3HOXPP2GRRA5TXQIWST42L54244CQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.955147294 (311h54m50.475971039s) [noise=true , address=193.149.164.69:4086 , piece_id=U63LKEGO2ZO6YRTQ3PKLF63VIJMMV7B6XXJH35FFQM3P6KGF7GQA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.956601288 (311h54m50.477425033s) [noise=true , address=167.17.76.155:4920 , piece_id=WXCJUEM5FJLPWGXL5Y3P5W4ZFIOUI3WVN2FVY5MVQCNITDVDCRHA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:41.281556869 (311h54m51.802380614s) [upload=298824]: Next returned nil segment
May 13 18:42:41.281569513 (311h54m51.802393258s) [upload=298824]: Waiting for error group managing segments...
May 13 18:42:41.372457758 (311h54m51.893281503s) [noise=true , address=77.90.31.14:4691 , piece_id=JFI56VQMCJZRDD2BZ3GOSKFLNVIVHEV6U3WK7MZ6LDU77ZGPOJNQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:12TxXc8GpnEATg9PMHcQxRFYUz7STpdzMuQsFXkiM9DgzchDRjJ, address:77.90.31.14:4691): protocol: rpc: dial tcp 77.90.31.14:4691: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
May 13 18:42:41.657118546 (311h54m52.177942291s) [noise=true , address=193.149.164.71:4294 , piece_id=JNC7X62V4WU7N64M5SMBYCFFJRUSEJUV2F5EET5MEQN3CL7REKWQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.669588933 (311h54m52.190412668s) [noise=true , address=167.17.76.155:4920 , piece_id=WXCJUEM5FJLPWGXL5Y3P5W4ZFIOUI3WVN2FVY5MVQCNITDVDCRHA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.673347525 (311h54m52.19417128s) [noise=true , address=167.17.72.71:4793 , piece_id=SAEMFXJYRFHIJDBZFYAR7AXLDPHAXWTSFSIVKATGZ5SLFKQNOYVQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.809908244 (311h54m52.33073199s) [noise=true , address=145.79.227.6:4277 , piece_id=I6VBYHUGW4ENFKM6PWFNDNGVDZCZTUPWSJLTGAW4OXB3GMEUS3EA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.855212685 (311h54m52.376036441s) [noise=true , address=145.79.227.47:4282 , piece_id=UZ4CAH5KR2XUABI6KFT4QIE3HOXPP2GRRA5TXQIWST42L54244CQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.859506299 (311h54m52.380330034s) [noise=true , address=145.79.227.50:4380 , piece_id=BOO6A7PMYBGEE4PT6QUXUDBEEIE2P6G3BAHZXDKQIW5BEROQE6QA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.942638354 (311h54m52.463462099s) [noise=true , address=167.17.72.70:4778 , piece_id=7754ZHKMHT7GREAJKTPJ6VUOGTNU243QFPMJNIPYMEJK4QREYD4A, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.270329925 (311h54m52.79115367s) [noise=true , address=193.149.164.68:4056 , piece_id=5XNXS6DFS7EOWWKYJ5BDVFDUJPGCYQWT4L3HNH5JBMXJDH67WLXQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.305229522 (311h54m52.826053277s) [noise=true , address=193.149.164.69:4086 , piece_id=U63LKEGO2ZO6YRTQ3PKLF63VIJMMV7B6XXJH35FFQM3P6KGF7GQA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.470504164 (311h54m52.991327909s) [noise=true , address=45.91.116.14:4220 , piece_id=EDKJQP6N2GL2XKRCKNZV5TOKQK24OCKYYCLVSSBH6YHBLHWAIPNQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.529729148 (311h54m53.050552894s) [noise=true , address=45.91.116.9:4279 , piece_id=URBFWN55CEWJ6NHVUB3SABQLY7LM7IQTQUIWW2RFJVDOHNUIJRFA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.530517203 (311h54m53.051340948s) [noise=true , address=45.91.116.10:4110 , piece_id=BC5TDQDVKMQFESE5KYRT2BSS62JKEZFX33Y2IXB5S76DGNXF352Q, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.536522942 (311h54m53.057346687s) [noise=true , address=45.91.116.4:4942 , piece_id=STFL2OELEKJASIEZFAJ7HVRTCKKVTKY5LK5S5CYSRXZUZLQEOQDQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:43:38.986230390 (311h55m49.507054135s) [noise=true , address=77.90.31.15:4722 , piece_id=FHQJJBOHMFGMJ3BKJJJBOPLVATIA2AIC64XPOE25WG6IP6NN55PA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:12vx19tv5KVo9mpt896aAivK3RCEEaVnt67oRipCoU7QLa4GYzM, address:77.90.31.15:4722): protocol: rpc: dial tcp 77.90.31.15:4722: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 13 18:43:38.986322155 (311h55m49.50714591s) [noise=true , address=77.90.31.17:4791 , piece_id=2AJ2EQNEN2EAEOOO7ZWMF6BZNR5LMPYT7OGK6XQJL6NKF7NQF26Q, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM, address:77.90.31.17:4791): protocol: context deadline exceeded; context deadline exceeded
May 13 18:43:38.986356991 (311h55m49.507180736s) [seg_pos={0 0}, upload=298824]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
May 13 18:43:39.237544444 (311h55m49.758368189s) [upload=298824]: Done uploading segments...
May 13 18:43:39.237578539 (311h55m49.758402284s) [upload=298824]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 6.069051627s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
6.403621879s trace:
https://jaeger.dev.storj.io/trace/61f95f9c55543be9
FAIL (2026-05-13 19:21:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
May 13 18:42:39.654115046 (311h54m50.174938791s) [upload=298824]: Starting upload
May 13 18:42:39.654237830 (311h54m50.175061595s) [upload=298824]: Done starting upload
May 13 18:42:39.654294318 (311h54m50.175118103s) [upload=298824]: Uploading segments...
May 13 18:42:39.654647201 (311h54m50.175470986s) [upload=298824]: Got next segment. Inline: false
May 13 18:42:39.654688680 (311h54m50.175512425s) [upload=298824]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 18:42:39.654695634 (311h54m50.175519379s) [upload=298824]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 18:42:39.943710770 (311h54m50.464534515s) [seg_pos={0 0}, upload=298824]: Begin upload segment...
May 13 18:42:39.943793989 (311h54m50.464617765s) [seg_pos={0 0}, upload=298824]: Done begin upload segment.
May 13 18:42:39.943823095 (311h54m50.46464685s) [noise=true , address=145.79.227.50:4380 , piece_id=BOO6A7PMYBGEE4PT6QUXUDBEEIE2P6G3BAHZXDKQIW5BEROQE6QA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.943840539 (311h54m50.464664284s) [noise=true , address=193.149.164.71:4294 , piece_id=JNC7X62V4WU7N64M5SMBYCFFJRUSEJUV2F5EET5MEQN3CL7REKWQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944027826 (311h54m50.464851571s) [noise=true , address=77.90.31.17:4791 , piece_id=2AJ2EQNEN2EAEOOO7ZWMF6BZNR5LMPYT7OGK6XQJL6NKF7NQF26Q, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944491761 (311h54m50.465315506s) [noise=true , address=193.149.164.68:4056 , piece_id=5XNXS6DFS7EOWWKYJ5BDVFDUJPGCYQWT4L3HNH5JBMXJDH67WLXQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944901293 (311h54m50.465725038s) [noise=true , address=145.79.227.6:4277 , piece_id=I6VBYHUGW4ENFKM6PWFNDNGVDZCZTUPWSJLTGAW4OXB3GMEUS3EA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.945324029 (311h54m50.466147774s) [noise=true , address=167.17.72.71:4793 , piece_id=SAEMFXJYRFHIJDBZFYAR7AXLDPHAXWTSFSIVKATGZ5SLFKQNOYVQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.945722740 (311h54m50.466546485s) [noise=true , address=45.91.116.9:4279 , piece_id=URBFWN55CEWJ6NHVUB3SABQLY7LM7IQTQUIWW2RFJVDOHNUIJRFA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.946124527 (311h54m50.466948282s) [noise=true , address=45.91.116.14:4220 , piece_id=EDKJQP6N2GL2XKRCKNZV5TOKQK24OCKYYCLVSSBH6YHBLHWAIPNQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.946494653 (311h54m50.467318398s) [noise=true , address=167.17.72.70:4778 , piece_id=7754ZHKMHT7GREAJKTPJ6VUOGTNU243QFPMJNIPYMEJK4QREYD4A, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.947965089 (311h54m50.468788834s) [noise=true , address=77.90.31.14:4691 , piece_id=JFI56VQMCJZRDD2BZ3GOSKFLNVIVHEV6U3WK7MZ6LDU77ZGPOJNQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.948256265 (311h54m50.46908001s) [noise=true , address=77.90.31.15:4722 , piece_id=FHQJJBOHMFGMJ3BKJJJBOPLVATIA2AIC64XPOE25WG6IP6NN55PA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.950216525 (311h54m50.47104027s) [noise=true , address=45.91.116.10:4110 , piece_id=BC5TDQDVKMQFESE5KYRT2BSS62JKEZFX33Y2IXB5S76DGNXF352Q, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.951870151 (311h54m50.472693896s) [noise=true , address=45.91.116.4:4942 , piece_id=STFL2OELEKJASIEZFAJ7HVRTCKKVTKY5LK5S5CYSRXZUZLQEOQDQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.953453131 (311h54m50.474276876s) [noise=true , address=145.79.227.47:4282 , piece_id=UZ4CAH5KR2XUABI6KFT4QIE3HOXPP2GRRA5TXQIWST42L54244CQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.955147294 (311h54m50.475971039s) [noise=true , address=193.149.164.69:4086 , piece_id=U63LKEGO2ZO6YRTQ3PKLF63VIJMMV7B6XXJH35FFQM3P6KGF7GQA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.956601288 (311h54m50.477425033s) [noise=true , address=167.17.76.155:4920 , piece_id=WXCJUEM5FJLPWGXL5Y3P5W4ZFIOUI3WVN2FVY5MVQCNITDVDCRHA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:41.281556869 (311h54m51.802380614s) [upload=298824]: Next returned nil segment
May 13 18:42:41.281569513 (311h54m51.802393258s) [upload=298824]: Waiting for error group managing segments...
May 13 18:42:41.372457758 (311h54m51.893281503s) [noise=true , address=77.90.31.14:4691 , piece_id=JFI56VQMCJZRDD2BZ3GOSKFLNVIVHEV6U3WK7MZ6LDU77ZGPOJNQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:12TxXc8GpnEATg9PMHcQxRFYUz7STpdzMuQsFXkiM9DgzchDRjJ, address:77.90.31.14:4691): protocol: rpc: dial tcp 77.90.31.14:4691: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
May 13 18:42:41.657118546 (311h54m52.177942291s) [noise=true , address=193.149.164.71:4294 , piece_id=JNC7X62V4WU7N64M5SMBYCFFJRUSEJUV2F5EET5MEQN3CL7REKWQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.669588933 (311h54m52.190412668s) [noise=true , address=167.17.76.155:4920 , piece_id=WXCJUEM5FJLPWGXL5Y3P5W4ZFIOUI3WVN2FVY5MVQCNITDVDCRHA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.673347525 (311h54m52.19417128s) [noise=true , address=167.17.72.71:4793 , piece_id=SAEMFXJYRFHIJDBZFYAR7AXLDPHAXWTSFSIVKATGZ5SLFKQNOYVQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.809908244 (311h54m52.33073199s) [noise=true , address=145.79.227.6:4277 , piece_id=I6VBYHUGW4ENFKM6PWFNDNGVDZCZTUPWSJLTGAW4OXB3GMEUS3EA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.855212685 (311h54m52.376036441s) [noise=true , address=145.79.227.47:4282 , piece_id=UZ4CAH5KR2XUABI6KFT4QIE3HOXPP2GRRA5TXQIWST42L54244CQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.859506299 (311h54m52.380330034s) [noise=true , address=145.79.227.50:4380 , piece_id=BOO6A7PMYBGEE4PT6QUXUDBEEIE2P6G3BAHZXDKQIW5BEROQE6QA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.942638354 (311h54m52.463462099s) [noise=true , address=167.17.72.70:4778 , piece_id=7754ZHKMHT7GREAJKTPJ6VUOGTNU243QFPMJNIPYMEJK4QREYD4A, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.270329925 (311h54m52.79115367s) [noise=true , address=193.149.164.68:4056 , piece_id=5XNXS6DFS7EOWWKYJ5BDVFDUJPGCYQWT4L3HNH5JBMXJDH67WLXQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.305229522 (311h54m52.826053277s) [noise=true , address=193.149.164.69:4086 , piece_id=U63LKEGO2ZO6YRTQ3PKLF63VIJMMV7B6XXJH35FFQM3P6KGF7GQA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.470504164 (311h54m52.991327909s) [noise=true , address=45.91.116.14:4220 , piece_id=EDKJQP6N2GL2XKRCKNZV5TOKQK24OCKYYCLVSSBH6YHBLHWAIPNQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.529729148 (311h54m53.050552894s) [noise=true , address=45.91.116.9:4279 , piece_id=URBFWN55CEWJ6NHVUB3SABQLY7LM7IQTQUIWW2RFJVDOHNUIJRFA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.530517203 (311h54m53.051340948s) [noise=true , address=45.91.116.10:4110 , piece_id=BC5TDQDVKMQFESE5KYRT2BSS62JKEZFX33Y2IXB5S76DGNXF352Q, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.536522942 (311h54m53.057346687s) [noise=true , address=45.91.116.4:4942 , piece_id=STFL2OELEKJASIEZFAJ7HVRTCKKVTKY5LK5S5CYSRXZUZLQEOQDQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:43:38.986230390 (311h55m49.507054135s) [noise=true , address=77.90.31.15:4722 , piece_id=FHQJJBOHMFGMJ3BKJJJBOPLVATIA2AIC64XPOE25WG6IP6NN55PA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:12vx19tv5KVo9mpt896aAivK3RCEEaVnt67oRipCoU7QLa4GYzM, address:77.90.31.15:4722): protocol: rpc: dial tcp 77.90.31.15:4722: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 13 18:43:38.986322155 (311h55m49.50714591s) [noise=true , address=77.90.31.17:4791 , piece_id=2AJ2EQNEN2EAEOOO7ZWMF6BZNR5LMPYT7OGK6XQJL6NKF7NQF26Q, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM, address:77.90.31.17:4791): protocol: context deadline exceeded; context deadline exceeded
May 13 18:43:38.986356991 (311h55m49.507180736s) [seg_pos={0 0}, upload=298824]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
May 13 18:43:39.237544444 (311h55m49.758368189s) [upload=298824]: Done uploading segments...
May 13 18:43:39.237578539 (311h55m49.758402284s) [upload=298824]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 6.403644291s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...............................X...X............X...........] duration:
6.72418748s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:21:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.724214653s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [................................X...X............X..........] duration:
6.932961713s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:20:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.932985398s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
5.845326061s trace:
https://jaeger.dev.storj.io/trace/61f95f9c55543be9
FAIL (2026-05-13 19:20:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
May 13 18:42:39.654115046 (311h54m50.174938791s) [upload=298824]: Starting upload
May 13 18:42:39.654237830 (311h54m50.175061595s) [upload=298824]: Done starting upload
May 13 18:42:39.654294318 (311h54m50.175118103s) [upload=298824]: Uploading segments...
May 13 18:42:39.654647201 (311h54m50.175470986s) [upload=298824]: Got next segment. Inline: false
May 13 18:42:39.654688680 (311h54m50.175512425s) [upload=298824]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 18:42:39.654695634 (311h54m50.175519379s) [upload=298824]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 18:42:39.943710770 (311h54m50.464534515s) [seg_pos={0 0}, upload=298824]: Begin upload segment...
May 13 18:42:39.943793989 (311h54m50.464617765s) [seg_pos={0 0}, upload=298824]: Done begin upload segment.
May 13 18:42:39.943823095 (311h54m50.46464685s) [noise=true , address=145.79.227.50:4380 , piece_id=BOO6A7PMYBGEE4PT6QUXUDBEEIE2P6G3BAHZXDKQIW5BEROQE6QA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.943840539 (311h54m50.464664284s) [noise=true , address=193.149.164.71:4294 , piece_id=JNC7X62V4WU7N64M5SMBYCFFJRUSEJUV2F5EET5MEQN3CL7REKWQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944027826 (311h54m50.464851571s) [noise=true , address=77.90.31.17:4791 , piece_id=2AJ2EQNEN2EAEOOO7ZWMF6BZNR5LMPYT7OGK6XQJL6NKF7NQF26Q, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944491761 (311h54m50.465315506s) [noise=true , address=193.149.164.68:4056 , piece_id=5XNXS6DFS7EOWWKYJ5BDVFDUJPGCYQWT4L3HNH5JBMXJDH67WLXQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944901293 (311h54m50.465725038s) [noise=true , address=145.79.227.6:4277 , piece_id=I6VBYHUGW4ENFKM6PWFNDNGVDZCZTUPWSJLTGAW4OXB3GMEUS3EA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.945324029 (311h54m50.466147774s) [noise=true , address=167.17.72.71:4793 , piece_id=SAEMFXJYRFHIJDBZFYAR7AXLDPHAXWTSFSIVKATGZ5SLFKQNOYVQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.945722740 (311h54m50.466546485s) [noise=true , address=45.91.116.9:4279 , piece_id=URBFWN55CEWJ6NHVUB3SABQLY7LM7IQTQUIWW2RFJVDOHNUIJRFA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.946124527 (311h54m50.466948282s) [noise=true , address=45.91.116.14:4220 , piece_id=EDKJQP6N2GL2XKRCKNZV5TOKQK24OCKYYCLVSSBH6YHBLHWAIPNQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.946494653 (311h54m50.467318398s) [noise=true , address=167.17.72.70:4778 , piece_id=7754ZHKMHT7GREAJKTPJ6VUOGTNU243QFPMJNIPYMEJK4QREYD4A, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.947965089 (311h54m50.468788834s) [noise=true , address=77.90.31.14:4691 , piece_id=JFI56VQMCJZRDD2BZ3GOSKFLNVIVHEV6U3WK7MZ6LDU77ZGPOJNQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.948256265 (311h54m50.46908001s) [noise=true , address=77.90.31.15:4722 , piece_id=FHQJJBOHMFGMJ3BKJJJBOPLVATIA2AIC64XPOE25WG6IP6NN55PA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.950216525 (311h54m50.47104027s) [noise=true , address=45.91.116.10:4110 , piece_id=BC5TDQDVKMQFESE5KYRT2BSS62JKEZFX33Y2IXB5S76DGNXF352Q, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.951870151 (311h54m50.472693896s) [noise=true , address=45.91.116.4:4942 , piece_id=STFL2OELEKJASIEZFAJ7HVRTCKKVTKY5LK5S5CYSRXZUZLQEOQDQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.953453131 (311h54m50.474276876s) [noise=true , address=145.79.227.47:4282 , piece_id=UZ4CAH5KR2XUABI6KFT4QIE3HOXPP2GRRA5TXQIWST42L54244CQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.955147294 (311h54m50.475971039s) [noise=true , address=193.149.164.69:4086 , piece_id=U63LKEGO2ZO6YRTQ3PKLF63VIJMMV7B6XXJH35FFQM3P6KGF7GQA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.956601288 (311h54m50.477425033s) [noise=true , address=167.17.76.155:4920 , piece_id=WXCJUEM5FJLPWGXL5Y3P5W4ZFIOUI3WVN2FVY5MVQCNITDVDCRHA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:41.281556869 (311h54m51.802380614s) [upload=298824]: Next returned nil segment
May 13 18:42:41.281569513 (311h54m51.802393258s) [upload=298824]: Waiting for error group managing segments...
May 13 18:42:41.372457758 (311h54m51.893281503s) [noise=true , address=77.90.31.14:4691 , piece_id=JFI56VQMCJZRDD2BZ3GOSKFLNVIVHEV6U3WK7MZ6LDU77ZGPOJNQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:12TxXc8GpnEATg9PMHcQxRFYUz7STpdzMuQsFXkiM9DgzchDRjJ, address:77.90.31.14:4691): protocol: rpc: dial tcp 77.90.31.14:4691: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
May 13 18:42:41.657118546 (311h54m52.177942291s) [noise=true , address=193.149.164.71:4294 , piece_id=JNC7X62V4WU7N64M5SMBYCFFJRUSEJUV2F5EET5MEQN3CL7REKWQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.669588933 (311h54m52.190412668s) [noise=true , address=167.17.76.155:4920 , piece_id=WXCJUEM5FJLPWGXL5Y3P5W4ZFIOUI3WVN2FVY5MVQCNITDVDCRHA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.673347525 (311h54m52.19417128s) [noise=true , address=167.17.72.71:4793 , piece_id=SAEMFXJYRFHIJDBZFYAR7AXLDPHAXWTSFSIVKATGZ5SLFKQNOYVQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.809908244 (311h54m52.33073199s) [noise=true , address=145.79.227.6:4277 , piece_id=I6VBYHUGW4ENFKM6PWFNDNGVDZCZTUPWSJLTGAW4OXB3GMEUS3EA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.855212685 (311h54m52.376036441s) [noise=true , address=145.79.227.47:4282 , piece_id=UZ4CAH5KR2XUABI6KFT4QIE3HOXPP2GRRA5TXQIWST42L54244CQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.859506299 (311h54m52.380330034s) [noise=true , address=145.79.227.50:4380 , piece_id=BOO6A7PMYBGEE4PT6QUXUDBEEIE2P6G3BAHZXDKQIW5BEROQE6QA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.942638354 (311h54m52.463462099s) [noise=true , address=167.17.72.70:4778 , piece_id=7754ZHKMHT7GREAJKTPJ6VUOGTNU243QFPMJNIPYMEJK4QREYD4A, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.270329925 (311h54m52.79115367s) [noise=true , address=193.149.164.68:4056 , piece_id=5XNXS6DFS7EOWWKYJ5BDVFDUJPGCYQWT4L3HNH5JBMXJDH67WLXQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.305229522 (311h54m52.826053277s) [noise=true , address=193.149.164.69:4086 , piece_id=U63LKEGO2ZO6YRTQ3PKLF63VIJMMV7B6XXJH35FFQM3P6KGF7GQA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.470504164 (311h54m52.991327909s) [noise=true , address=45.91.116.14:4220 , piece_id=EDKJQP6N2GL2XKRCKNZV5TOKQK24OCKYYCLVSSBH6YHBLHWAIPNQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.529729148 (311h54m53.050552894s) [noise=true , address=45.91.116.9:4279 , piece_id=URBFWN55CEWJ6NHVUB3SABQLY7LM7IQTQUIWW2RFJVDOHNUIJRFA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.530517203 (311h54m53.051340948s) [noise=true , address=45.91.116.10:4110 , piece_id=BC5TDQDVKMQFESE5KYRT2BSS62JKEZFX33Y2IXB5S76DGNXF352Q, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.536522942 (311h54m53.057346687s) [noise=true , address=45.91.116.4:4942 , piece_id=STFL2OELEKJASIEZFAJ7HVRTCKKVTKY5LK5S5CYSRXZUZLQEOQDQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:43:38.986230390 (311h55m49.507054135s) [noise=true , address=77.90.31.15:4722 , piece_id=FHQJJBOHMFGMJ3BKJJJBOPLVATIA2AIC64XPOE25WG6IP6NN55PA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:12vx19tv5KVo9mpt896aAivK3RCEEaVnt67oRipCoU7QLa4GYzM, address:77.90.31.15:4722): protocol: rpc: dial tcp 77.90.31.15:4722: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 13 18:43:38.986322155 (311h55m49.50714591s) [noise=true , address=77.90.31.17:4791 , piece_id=2AJ2EQNEN2EAEOOO7ZWMF6BZNR5LMPYT7OGK6XQJL6NKF7NQF26Q, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM, address:77.90.31.17:4791): protocol: context deadline exceeded; context deadline exceeded
May 13 18:43:38.986356991 (311h55m49.507180736s) [seg_pos={0 0}, upload=298824]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
May 13 18:43:39.237544444 (311h55m49.758368189s) [upload=298824]: Done uploading segments...
May 13 18:43:39.237578539 (311h55m49.758402284s) [upload=298824]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 5.845350286s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.................................X...X............X.........] duration:
6.854754291s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:20:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.854766444s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..................................X...X............X........] duration:
38.548563937s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:20:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 38.548581129s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
7.459453323s trace:
https://jaeger.dev.storj.io/trace/61f95f9c55543be9
FAIL (2026-05-13 19:19:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
May 13 18:42:39.654115046 (311h54m50.174938791s) [upload=298824]: Starting upload
May 13 18:42:39.654237830 (311h54m50.175061595s) [upload=298824]: Done starting upload
May 13 18:42:39.654294318 (311h54m50.175118103s) [upload=298824]: Uploading segments...
May 13 18:42:39.654647201 (311h54m50.175470986s) [upload=298824]: Got next segment. Inline: false
May 13 18:42:39.654688680 (311h54m50.175512425s) [upload=298824]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 18:42:39.654695634 (311h54m50.175519379s) [upload=298824]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 18:42:39.943710770 (311h54m50.464534515s) [seg_pos={0 0}, upload=298824]: Begin upload segment...
May 13 18:42:39.943793989 (311h54m50.464617765s) [seg_pos={0 0}, upload=298824]: Done begin upload segment.
May 13 18:42:39.943823095 (311h54m50.46464685s) [noise=true , address=145.79.227.50:4380 , piece_id=BOO6A7PMYBGEE4PT6QUXUDBEEIE2P6G3BAHZXDKQIW5BEROQE6QA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.943840539 (311h54m50.464664284s) [noise=true , address=193.149.164.71:4294 , piece_id=JNC7X62V4WU7N64M5SMBYCFFJRUSEJUV2F5EET5MEQN3CL7REKWQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944027826 (311h54m50.464851571s) [noise=true , address=77.90.31.17:4791 , piece_id=2AJ2EQNEN2EAEOOO7ZWMF6BZNR5LMPYT7OGK6XQJL6NKF7NQF26Q, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944491761 (311h54m50.465315506s) [noise=true , address=193.149.164.68:4056 , piece_id=5XNXS6DFS7EOWWKYJ5BDVFDUJPGCYQWT4L3HNH5JBMXJDH67WLXQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944901293 (311h54m50.465725038s) [noise=true , address=145.79.227.6:4277 , piece_id=I6VBYHUGW4ENFKM6PWFNDNGVDZCZTUPWSJLTGAW4OXB3GMEUS3EA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.945324029 (311h54m50.466147774s) [noise=true , address=167.17.72.71:4793 , piece_id=SAEMFXJYRFHIJDBZFYAR7AXLDPHAXWTSFSIVKATGZ5SLFKQNOYVQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.945722740 (311h54m50.466546485s) [noise=true , address=45.91.116.9:4279 , piece_id=URBFWN55CEWJ6NHVUB3SABQLY7LM7IQTQUIWW2RFJVDOHNUIJRFA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.946124527 (311h54m50.466948282s) [noise=true , address=45.91.116.14:4220 , piece_id=EDKJQP6N2GL2XKRCKNZV5TOKQK24OCKYYCLVSSBH6YHBLHWAIPNQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.946494653 (311h54m50.467318398s) [noise=true , address=167.17.72.70:4778 , piece_id=7754ZHKMHT7GREAJKTPJ6VUOGTNU243QFPMJNIPYMEJK4QREYD4A, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.947965089 (311h54m50.468788834s) [noise=true , address=77.90.31.14:4691 , piece_id=JFI56VQMCJZRDD2BZ3GOSKFLNVIVHEV6U3WK7MZ6LDU77ZGPOJNQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.948256265 (311h54m50.46908001s) [noise=true , address=77.90.31.15:4722 , piece_id=FHQJJBOHMFGMJ3BKJJJBOPLVATIA2AIC64XPOE25WG6IP6NN55PA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.950216525 (311h54m50.47104027s) [noise=true , address=45.91.116.10:4110 , piece_id=BC5TDQDVKMQFESE5KYRT2BSS62JKEZFX33Y2IXB5S76DGNXF352Q, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.951870151 (311h54m50.472693896s) [noise=true , address=45.91.116.4:4942 , piece_id=STFL2OELEKJASIEZFAJ7HVRTCKKVTKY5LK5S5CYSRXZUZLQEOQDQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.953453131 (311h54m50.474276876s) [noise=true , address=145.79.227.47:4282 , piece_id=UZ4CAH5KR2XUABI6KFT4QIE3HOXPP2GRRA5TXQIWST42L54244CQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.955147294 (311h54m50.475971039s) [noise=true , address=193.149.164.69:4086 , piece_id=U63LKEGO2ZO6YRTQ3PKLF63VIJMMV7B6XXJH35FFQM3P6KGF7GQA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.956601288 (311h54m50.477425033s) [noise=true , address=167.17.76.155:4920 , piece_id=WXCJUEM5FJLPWGXL5Y3P5W4ZFIOUI3WVN2FVY5MVQCNITDVDCRHA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:41.281556869 (311h54m51.802380614s) [upload=298824]: Next returned nil segment
May 13 18:42:41.281569513 (311h54m51.802393258s) [upload=298824]: Waiting for error group managing segments...
May 13 18:42:41.372457758 (311h54m51.893281503s) [noise=true , address=77.90.31.14:4691 , piece_id=JFI56VQMCJZRDD2BZ3GOSKFLNVIVHEV6U3WK7MZ6LDU77ZGPOJNQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:12TxXc8GpnEATg9PMHcQxRFYUz7STpdzMuQsFXkiM9DgzchDRjJ, address:77.90.31.14:4691): protocol: rpc: dial tcp 77.90.31.14:4691: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
May 13 18:42:41.657118546 (311h54m52.177942291s) [noise=true , address=193.149.164.71:4294 , piece_id=JNC7X62V4WU7N64M5SMBYCFFJRUSEJUV2F5EET5MEQN3CL7REKWQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.669588933 (311h54m52.190412668s) [noise=true , address=167.17.76.155:4920 , piece_id=WXCJUEM5FJLPWGXL5Y3P5W4ZFIOUI3WVN2FVY5MVQCNITDVDCRHA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.673347525 (311h54m52.19417128s) [noise=true , address=167.17.72.71:4793 , piece_id=SAEMFXJYRFHIJDBZFYAR7AXLDPHAXWTSFSIVKATGZ5SLFKQNOYVQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.809908244 (311h54m52.33073199s) [noise=true , address=145.79.227.6:4277 , piece_id=I6VBYHUGW4ENFKM6PWFNDNGVDZCZTUPWSJLTGAW4OXB3GMEUS3EA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.855212685 (311h54m52.376036441s) [noise=true , address=145.79.227.47:4282 , piece_id=UZ4CAH5KR2XUABI6KFT4QIE3HOXPP2GRRA5TXQIWST42L54244CQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.859506299 (311h54m52.380330034s) [noise=true , address=145.79.227.50:4380 , piece_id=BOO6A7PMYBGEE4PT6QUXUDBEEIE2P6G3BAHZXDKQIW5BEROQE6QA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.942638354 (311h54m52.463462099s) [noise=true , address=167.17.72.70:4778 , piece_id=7754ZHKMHT7GREAJKTPJ6VUOGTNU243QFPMJNIPYMEJK4QREYD4A, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.270329925 (311h54m52.79115367s) [noise=true , address=193.149.164.68:4056 , piece_id=5XNXS6DFS7EOWWKYJ5BDVFDUJPGCYQWT4L3HNH5JBMXJDH67WLXQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.305229522 (311h54m52.826053277s) [noise=true , address=193.149.164.69:4086 , piece_id=U63LKEGO2ZO6YRTQ3PKLF63VIJMMV7B6XXJH35FFQM3P6KGF7GQA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.470504164 (311h54m52.991327909s) [noise=true , address=45.91.116.14:4220 , piece_id=EDKJQP6N2GL2XKRCKNZV5TOKQK24OCKYYCLVSSBH6YHBLHWAIPNQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.529729148 (311h54m53.050552894s) [noise=true , address=45.91.116.9:4279 , piece_id=URBFWN55CEWJ6NHVUB3SABQLY7LM7IQTQUIWW2RFJVDOHNUIJRFA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.530517203 (311h54m53.051340948s) [noise=true , address=45.91.116.10:4110 , piece_id=BC5TDQDVKMQFESE5KYRT2BSS62JKEZFX33Y2IXB5S76DGNXF352Q, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.536522942 (311h54m53.057346687s) [noise=true , address=45.91.116.4:4942 , piece_id=STFL2OELEKJASIEZFAJ7HVRTCKKVTKY5LK5S5CYSRXZUZLQEOQDQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:43:38.986230390 (311h55m49.507054135s) [noise=true , address=77.90.31.15:4722 , piece_id=FHQJJBOHMFGMJ3BKJJJBOPLVATIA2AIC64XPOE25WG6IP6NN55PA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:12vx19tv5KVo9mpt896aAivK3RCEEaVnt67oRipCoU7QLa4GYzM, address:77.90.31.15:4722): protocol: rpc: dial tcp 77.90.31.15:4722: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 13 18:43:38.986322155 (311h55m49.50714591s) [noise=true , address=77.90.31.17:4791 , piece_id=2AJ2EQNEN2EAEOOO7ZWMF6BZNR5LMPYT7OGK6XQJL6NKF7NQF26Q, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM, address:77.90.31.17:4791): protocol: context deadline exceeded; context deadline exceeded
May 13 18:43:38.986356991 (311h55m49.507180736s) [seg_pos={0 0}, upload=298824]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
May 13 18:43:39.237544444 (311h55m49.758368189s) [upload=298824]: Done uploading segments...
May 13 18:43:39.237578539 (311h55m49.758402284s) [upload=298824]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 7.45947794s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
7.535223833s trace:
https://jaeger.dev.storj.io/trace/61f95f9c55543be9
FAIL (2026-05-13 19:18:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
May 13 18:42:39.654115046 (311h54m50.174938791s) [upload=298824]: Starting upload
May 13 18:42:39.654237830 (311h54m50.175061595s) [upload=298824]: Done starting upload
May 13 18:42:39.654294318 (311h54m50.175118103s) [upload=298824]: Uploading segments...
May 13 18:42:39.654647201 (311h54m50.175470986s) [upload=298824]: Got next segment. Inline: false
May 13 18:42:39.654688680 (311h54m50.175512425s) [upload=298824]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 18:42:39.654695634 (311h54m50.175519379s) [upload=298824]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 18:42:39.943710770 (311h54m50.464534515s) [seg_pos={0 0}, upload=298824]: Begin upload segment...
May 13 18:42:39.943793989 (311h54m50.464617765s) [seg_pos={0 0}, upload=298824]: Done begin upload segment.
May 13 18:42:39.943823095 (311h54m50.46464685s) [noise=true , address=145.79.227.50:4380 , piece_id=BOO6A7PMYBGEE4PT6QUXUDBEEIE2P6G3BAHZXDKQIW5BEROQE6QA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.943840539 (311h54m50.464664284s) [noise=true , address=193.149.164.71:4294 , piece_id=JNC7X62V4WU7N64M5SMBYCFFJRUSEJUV2F5EET5MEQN3CL7REKWQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944027826 (311h54m50.464851571s) [noise=true , address=77.90.31.17:4791 , piece_id=2AJ2EQNEN2EAEOOO7ZWMF6BZNR5LMPYT7OGK6XQJL6NKF7NQF26Q, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944491761 (311h54m50.465315506s) [noise=true , address=193.149.164.68:4056 , piece_id=5XNXS6DFS7EOWWKYJ5BDVFDUJPGCYQWT4L3HNH5JBMXJDH67WLXQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944901293 (311h54m50.465725038s) [noise=true , address=145.79.227.6:4277 , piece_id=I6VBYHUGW4ENFKM6PWFNDNGVDZCZTUPWSJLTGAW4OXB3GMEUS3EA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.945324029 (311h54m50.466147774s) [noise=true , address=167.17.72.71:4793 , piece_id=SAEMFXJYRFHIJDBZFYAR7AXLDPHAXWTSFSIVKATGZ5SLFKQNOYVQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.945722740 (311h54m50.466546485s) [noise=true , address=45.91.116.9:4279 , piece_id=URBFWN55CEWJ6NHVUB3SABQLY7LM7IQTQUIWW2RFJVDOHNUIJRFA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.946124527 (311h54m50.466948282s) [noise=true , address=45.91.116.14:4220 , piece_id=EDKJQP6N2GL2XKRCKNZV5TOKQK24OCKYYCLVSSBH6YHBLHWAIPNQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.946494653 (311h54m50.467318398s) [noise=true , address=167.17.72.70:4778 , piece_id=7754ZHKMHT7GREAJKTPJ6VUOGTNU243QFPMJNIPYMEJK4QREYD4A, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.947965089 (311h54m50.468788834s) [noise=true , address=77.90.31.14:4691 , piece_id=JFI56VQMCJZRDD2BZ3GOSKFLNVIVHEV6U3WK7MZ6LDU77ZGPOJNQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.948256265 (311h54m50.46908001s) [noise=true , address=77.90.31.15:4722 , piece_id=FHQJJBOHMFGMJ3BKJJJBOPLVATIA2AIC64XPOE25WG6IP6NN55PA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.950216525 (311h54m50.47104027s) [noise=true , address=45.91.116.10:4110 , piece_id=BC5TDQDVKMQFESE5KYRT2BSS62JKEZFX33Y2IXB5S76DGNXF352Q, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.951870151 (311h54m50.472693896s) [noise=true , address=45.91.116.4:4942 , piece_id=STFL2OELEKJASIEZFAJ7HVRTCKKVTKY5LK5S5CYSRXZUZLQEOQDQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.953453131 (311h54m50.474276876s) [noise=true , address=145.79.227.47:4282 , piece_id=UZ4CAH5KR2XUABI6KFT4QIE3HOXPP2GRRA5TXQIWST42L54244CQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.955147294 (311h54m50.475971039s) [noise=true , address=193.149.164.69:4086 , piece_id=U63LKEGO2ZO6YRTQ3PKLF63VIJMMV7B6XXJH35FFQM3P6KGF7GQA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.956601288 (311h54m50.477425033s) [noise=true , address=167.17.76.155:4920 , piece_id=WXCJUEM5FJLPWGXL5Y3P5W4ZFIOUI3WVN2FVY5MVQCNITDVDCRHA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:41.281556869 (311h54m51.802380614s) [upload=298824]: Next returned nil segment
May 13 18:42:41.281569513 (311h54m51.802393258s) [upload=298824]: Waiting for error group managing segments...
May 13 18:42:41.372457758 (311h54m51.893281503s) [noise=true , address=77.90.31.14:4691 , piece_id=JFI56VQMCJZRDD2BZ3GOSKFLNVIVHEV6U3WK7MZ6LDU77ZGPOJNQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:12TxXc8GpnEATg9PMHcQxRFYUz7STpdzMuQsFXkiM9DgzchDRjJ, address:77.90.31.14:4691): protocol: rpc: dial tcp 77.90.31.14:4691: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
May 13 18:42:41.657118546 (311h54m52.177942291s) [noise=true , address=193.149.164.71:4294 , piece_id=JNC7X62V4WU7N64M5SMBYCFFJRUSEJUV2F5EET5MEQN3CL7REKWQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.669588933 (311h54m52.190412668s) [noise=true , address=167.17.76.155:4920 , piece_id=WXCJUEM5FJLPWGXL5Y3P5W4ZFIOUI3WVN2FVY5MVQCNITDVDCRHA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.673347525 (311h54m52.19417128s) [noise=true , address=167.17.72.71:4793 , piece_id=SAEMFXJYRFHIJDBZFYAR7AXLDPHAXWTSFSIVKATGZ5SLFKQNOYVQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.809908244 (311h54m52.33073199s) [noise=true , address=145.79.227.6:4277 , piece_id=I6VBYHUGW4ENFKM6PWFNDNGVDZCZTUPWSJLTGAW4OXB3GMEUS3EA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.855212685 (311h54m52.376036441s) [noise=true , address=145.79.227.47:4282 , piece_id=UZ4CAH5KR2XUABI6KFT4QIE3HOXPP2GRRA5TXQIWST42L54244CQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.859506299 (311h54m52.380330034s) [noise=true , address=145.79.227.50:4380 , piece_id=BOO6A7PMYBGEE4PT6QUXUDBEEIE2P6G3BAHZXDKQIW5BEROQE6QA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.942638354 (311h54m52.463462099s) [noise=true , address=167.17.72.70:4778 , piece_id=7754ZHKMHT7GREAJKTPJ6VUOGTNU243QFPMJNIPYMEJK4QREYD4A, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.270329925 (311h54m52.79115367s) [noise=true , address=193.149.164.68:4056 , piece_id=5XNXS6DFS7EOWWKYJ5BDVFDUJPGCYQWT4L3HNH5JBMXJDH67WLXQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.305229522 (311h54m52.826053277s) [noise=true , address=193.149.164.69:4086 , piece_id=U63LKEGO2ZO6YRTQ3PKLF63VIJMMV7B6XXJH35FFQM3P6KGF7GQA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.470504164 (311h54m52.991327909s) [noise=true , address=45.91.116.14:4220 , piece_id=EDKJQP6N2GL2XKRCKNZV5TOKQK24OCKYYCLVSSBH6YHBLHWAIPNQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.529729148 (311h54m53.050552894s) [noise=true , address=45.91.116.9:4279 , piece_id=URBFWN55CEWJ6NHVUB3SABQLY7LM7IQTQUIWW2RFJVDOHNUIJRFA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.530517203 (311h54m53.051340948s) [noise=true , address=45.91.116.10:4110 , piece_id=BC5TDQDVKMQFESE5KYRT2BSS62JKEZFX33Y2IXB5S76DGNXF352Q, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.536522942 (311h54m53.057346687s) [noise=true , address=45.91.116.4:4942 , piece_id=STFL2OELEKJASIEZFAJ7HVRTCKKVTKY5LK5S5CYSRXZUZLQEOQDQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:43:38.986230390 (311h55m49.507054135s) [noise=true , address=77.90.31.15:4722 , piece_id=FHQJJBOHMFGMJ3BKJJJBOPLVATIA2AIC64XPOE25WG6IP6NN55PA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:12vx19tv5KVo9mpt896aAivK3RCEEaVnt67oRipCoU7QLa4GYzM, address:77.90.31.15:4722): protocol: rpc: dial tcp 77.90.31.15:4722: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 13 18:43:38.986322155 (311h55m49.50714591s) [noise=true , address=77.90.31.17:4791 , piece_id=2AJ2EQNEN2EAEOOO7ZWMF6BZNR5LMPYT7OGK6XQJL6NKF7NQF26Q, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM, address:77.90.31.17:4791): protocol: context deadline exceeded; context deadline exceeded
May 13 18:43:38.986356991 (311h55m49.507180736s) [seg_pos={0 0}, upload=298824]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
May 13 18:43:39.237544444 (311h55m49.758368189s) [upload=298824]: Done uploading segments...
May 13 18:43:39.237578539 (311h55m49.758402284s) [upload=298824]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 7.535245805s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...................................X...X............X.......] duration:
16.128052169s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:18:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 16.128077787s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
6.399332641s trace:
https://jaeger.dev.storj.io/trace/61f95f9c55543be9
FAIL (2026-05-13 19:18:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
May 13 18:42:39.654115046 (311h54m50.174938791s) [upload=298824]: Starting upload
May 13 18:42:39.654237830 (311h54m50.175061595s) [upload=298824]: Done starting upload
May 13 18:42:39.654294318 (311h54m50.175118103s) [upload=298824]: Uploading segments...
May 13 18:42:39.654647201 (311h54m50.175470986s) [upload=298824]: Got next segment. Inline: false
May 13 18:42:39.654688680 (311h54m50.175512425s) [upload=298824]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 13 18:42:39.654695634 (311h54m50.175519379s) [upload=298824]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 13 18:42:39.943710770 (311h54m50.464534515s) [seg_pos={0 0}, upload=298824]: Begin upload segment...
May 13 18:42:39.943793989 (311h54m50.464617765s) [seg_pos={0 0}, upload=298824]: Done begin upload segment.
May 13 18:42:39.943823095 (311h54m50.46464685s) [noise=true , address=145.79.227.50:4380 , piece_id=BOO6A7PMYBGEE4PT6QUXUDBEEIE2P6G3BAHZXDKQIW5BEROQE6QA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.943840539 (311h54m50.464664284s) [noise=true , address=193.149.164.71:4294 , piece_id=JNC7X62V4WU7N64M5SMBYCFFJRUSEJUV2F5EET5MEQN3CL7REKWQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944027826 (311h54m50.464851571s) [noise=true , address=77.90.31.17:4791 , piece_id=2AJ2EQNEN2EAEOOO7ZWMF6BZNR5LMPYT7OGK6XQJL6NKF7NQF26Q, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944491761 (311h54m50.465315506s) [noise=true , address=193.149.164.68:4056 , piece_id=5XNXS6DFS7EOWWKYJ5BDVFDUJPGCYQWT4L3HNH5JBMXJDH67WLXQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.944901293 (311h54m50.465725038s) [noise=true , address=145.79.227.6:4277 , piece_id=I6VBYHUGW4ENFKM6PWFNDNGVDZCZTUPWSJLTGAW4OXB3GMEUS3EA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.945324029 (311h54m50.466147774s) [noise=true , address=167.17.72.71:4793 , piece_id=SAEMFXJYRFHIJDBZFYAR7AXLDPHAXWTSFSIVKATGZ5SLFKQNOYVQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.945722740 (311h54m50.466546485s) [noise=true , address=45.91.116.9:4279 , piece_id=URBFWN55CEWJ6NHVUB3SABQLY7LM7IQTQUIWW2RFJVDOHNUIJRFA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.946124527 (311h54m50.466948282s) [noise=true , address=45.91.116.14:4220 , piece_id=EDKJQP6N2GL2XKRCKNZV5TOKQK24OCKYYCLVSSBH6YHBLHWAIPNQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.946494653 (311h54m50.467318398s) [noise=true , address=167.17.72.70:4778 , piece_id=7754ZHKMHT7GREAJKTPJ6VUOGTNU243QFPMJNIPYMEJK4QREYD4A, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.947965089 (311h54m50.468788834s) [noise=true , address=77.90.31.14:4691 , piece_id=JFI56VQMCJZRDD2BZ3GOSKFLNVIVHEV6U3WK7MZ6LDU77ZGPOJNQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.948256265 (311h54m50.46908001s) [noise=true , address=77.90.31.15:4722 , piece_id=FHQJJBOHMFGMJ3BKJJJBOPLVATIA2AIC64XPOE25WG6IP6NN55PA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.950216525 (311h54m50.47104027s) [noise=true , address=45.91.116.10:4110 , piece_id=BC5TDQDVKMQFESE5KYRT2BSS62JKEZFX33Y2IXB5S76DGNXF352Q, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.951870151 (311h54m50.472693896s) [noise=true , address=45.91.116.4:4942 , piece_id=STFL2OELEKJASIEZFAJ7HVRTCKKVTKY5LK5S5CYSRXZUZLQEOQDQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.953453131 (311h54m50.474276876s) [noise=true , address=145.79.227.47:4282 , piece_id=UZ4CAH5KR2XUABI6KFT4QIE3HOXPP2GRRA5TXQIWST42L54244CQ, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.955147294 (311h54m50.475971039s) [noise=true , address=193.149.164.69:4086 , piece_id=U63LKEGO2ZO6YRTQ3PKLF63VIJMMV7B6XXJH35FFQM3P6KGF7GQA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:39.956601288 (311h54m50.477425033s) [noise=true , address=167.17.76.155:4920 , piece_id=WXCJUEM5FJLPWGXL5Y3P5W4ZFIOUI3WVN2FVY5MVQCNITDVDCRHA, seg_pos={0 0}, upload=298824]: Uploading piece...
May 13 18:42:41.281556869 (311h54m51.802380614s) [upload=298824]: Next returned nil segment
May 13 18:42:41.281569513 (311h54m51.802393258s) [upload=298824]: Waiting for error group managing segments...
May 13 18:42:41.372457758 (311h54m51.893281503s) [noise=true , address=77.90.31.14:4691 , piece_id=JFI56VQMCJZRDD2BZ3GOSKFLNVIVHEV6U3WK7MZ6LDU77ZGPOJNQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:12TxXc8GpnEATg9PMHcQxRFYUz7STpdzMuQsFXkiM9DgzchDRjJ, address:77.90.31.14:4691): protocol: rpc: dial tcp 77.90.31.14:4691: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
May 13 18:42:41.657118546 (311h54m52.177942291s) [noise=true , address=193.149.164.71:4294 , piece_id=JNC7X62V4WU7N64M5SMBYCFFJRUSEJUV2F5EET5MEQN3CL7REKWQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.669588933 (311h54m52.190412668s) [noise=true , address=167.17.76.155:4920 , piece_id=WXCJUEM5FJLPWGXL5Y3P5W4ZFIOUI3WVN2FVY5MVQCNITDVDCRHA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.673347525 (311h54m52.19417128s) [noise=true , address=167.17.72.71:4793 , piece_id=SAEMFXJYRFHIJDBZFYAR7AXLDPHAXWTSFSIVKATGZ5SLFKQNOYVQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.809908244 (311h54m52.33073199s) [noise=true , address=145.79.227.6:4277 , piece_id=I6VBYHUGW4ENFKM6PWFNDNGVDZCZTUPWSJLTGAW4OXB3GMEUS3EA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.855212685 (311h54m52.376036441s) [noise=true , address=145.79.227.47:4282 , piece_id=UZ4CAH5KR2XUABI6KFT4QIE3HOXPP2GRRA5TXQIWST42L54244CQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.859506299 (311h54m52.380330034s) [noise=true , address=145.79.227.50:4380 , piece_id=BOO6A7PMYBGEE4PT6QUXUDBEEIE2P6G3BAHZXDKQIW5BEROQE6QA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:41.942638354 (311h54m52.463462099s) [noise=true , address=167.17.72.70:4778 , piece_id=7754ZHKMHT7GREAJKTPJ6VUOGTNU243QFPMJNIPYMEJK4QREYD4A, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.270329925 (311h54m52.79115367s) [noise=true , address=193.149.164.68:4056 , piece_id=5XNXS6DFS7EOWWKYJ5BDVFDUJPGCYQWT4L3HNH5JBMXJDH67WLXQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.305229522 (311h54m52.826053277s) [noise=true , address=193.149.164.69:4086 , piece_id=U63LKEGO2ZO6YRTQ3PKLF63VIJMMV7B6XXJH35FFQM3P6KGF7GQA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.470504164 (311h54m52.991327909s) [noise=true , address=45.91.116.14:4220 , piece_id=EDKJQP6N2GL2XKRCKNZV5TOKQK24OCKYYCLVSSBH6YHBLHWAIPNQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.529729148 (311h54m53.050552894s) [noise=true , address=45.91.116.9:4279 , piece_id=URBFWN55CEWJ6NHVUB3SABQLY7LM7IQTQUIWW2RFJVDOHNUIJRFA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.530517203 (311h54m53.051340948s) [noise=true , address=45.91.116.10:4110 , piece_id=BC5TDQDVKMQFESE5KYRT2BSS62JKEZFX33Y2IXB5S76DGNXF352Q, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:42:42.536522942 (311h54m53.057346687s) [noise=true , address=45.91.116.4:4942 , piece_id=STFL2OELEKJASIEZFAJ7HVRTCKKVTKY5LK5S5CYSRXZUZLQEOQDQ, seg_pos={0 0}, upload=298824]: Done uploading piece. err: <nil>
May 13 18:43:38.986230390 (311h55m49.507054135s) [noise=true , address=77.90.31.15:4722 , piece_id=FHQJJBOHMFGMJ3BKJJJBOPLVATIA2AIC64XPOE25WG6IP6NN55PA, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:12vx19tv5KVo9mpt896aAivK3RCEEaVnt67oRipCoU7QLa4GYzM, address:77.90.31.15:4722): protocol: rpc: dial tcp 77.90.31.15:4722: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 13 18:43:38.986322155 (311h55m49.50714591s) [noise=true , address=77.90.31.17:4791 , piece_id=2AJ2EQNEN2EAEOOO7ZWMF6BZNR5LMPYT7OGK6XQJL6NKF7NQF26Q, seg_pos={0 0}, upload=298824]: Done uploading piece. err: ecclient: upload failed (node:1bFEmP78qGhyHhXr3QNVptdiWQ4No4nrb31qj6DPgmJLcHYKiM, address:77.90.31.17:4791): protocol: context deadline exceeded; context deadline exceeded
May 13 18:43:38.986356991 (311h55m49.507180736s) [seg_pos={0 0}, upload=298824]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
May 13 18:43:39.237544444 (311h55m49.758368189s) [upload=298824]: Done uploading segments...
May 13 18:43:39.237578539 (311h55m49.758402284s) [upload=298824]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 6.39935804s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [....................................X...X............X......] duration:
7.407114836s trace:
https://jaeger.dev.storj.io/trace/7d4dd38dc29b2289
FAIL (2026-05-13 19:18:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.407140223s
core
uplink
us-select-1
us1-placement-20
|