Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [.....X....X...XX.X.XX.......XX.X.X..X...........X.X..X.X..XX] duration:
1m0.001723576s trace:
https://jaeger.dev.storj.io/trace/6a168d54ce121c63
FAIL (2026-05-15 23:26:00 UTC)
orbiter warning: context deadline exceeded
Duration: 1m0.001737462s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [......XX....XXX.......................X........X............] duration:
3.947832154s trace:
https://jaeger.dev.storj.io/trace/43cf5080575f2b7d
FAIL (2026-05-15 23:25:17 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.947872091s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
17 out of 60 is failed (threshold=3) [......X....X...XX.X.XX.......XX.X.X..X...........X.X..X.X..X] duration:
1m0.00147824s trace:
https://jaeger.dev.storj.io/trace/5be3051502e06750
FAIL (2026-05-15 23:25:00 UTC)
orbiter warning: context deadline exceeded
Duration: 1m0.001487036s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [.......XX....XXX.......................X........X...........] duration:
4.194562695s trace:
https://jaeger.dev.storj.io/trace/43cf5080575f2b7d
FAIL (2026-05-15 23:24:07 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.194589836s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
17 out of 60 is failed (threshold=3) [X......X....X...XX.X.XX.......XX.X.X..X...........X.X..X.X..] duration:
51.326436449s trace:
https://jaeger.dev.storj.io/trace/7ce89b4f0f1a1229
FAIL (2026-05-15 23:24:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 51.326840499s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [........XX....XXX.......................X........X..........] duration:
3.908002751s trace:
https://jaeger.dev.storj.io/trace/43cf5080575f2b7d
FAIL (2026-05-15 23:22:55 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.908065501s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
17 out of 60 is failed (threshold=3) [.X......X....X...XX.X.XX.......XX.X.X..X...........X.X..X.X.] duration:
12.561786808s trace:
https://jaeger.dev.storj.io/trace/7ce89b4f0f1a1229
FAIL (2026-05-15 23:22:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 12.561803448s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [.........XX....XXX.......................X........X.........] duration:
3.803428042s trace:
https://jaeger.dev.storj.io/trace/43cf5080575f2b7d
FAIL (2026-05-15 23:22:29 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.80345296s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [X.X......X....X...XX.X.XX.......XX.X.X..X...........X.X..X.X] duration:
1m0.00133979s trace:
https://jaeger.dev.storj.io/trace/7ce89b4f0f1a1229
FAIL (2026-05-15 23:22:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.00134982s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
17 out of 60 is failed (threshold=3) [.X.X......X....X...XX.X.XX.......XX.X.X..X...........X.X..X.] duration:
50.047653355s trace:
https://jaeger.dev.storj.io/trace/4cfff5e1870b3074
FAIL (2026-05-15 23:21:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 50.047664717s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [..........XX....XXX.......................X........X........] duration:
4.237630711s trace:
https://jaeger.dev.storj.io/trace/43cf5080575f2b7d
FAIL (2026-05-15 23:21:14 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.237652542s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [X.X.X......X....X...XX.X.XX.......XX.X.X..X...........X.X..X] duration:
1m0.000594998s trace:
https://jaeger.dev.storj.io/trace/4cfff5e1870b3074
FAIL (2026-05-15 23:20:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000639432s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [...........XX....XXX.......................X........X.......] duration:
3.90544875s trace:
https://jaeger.dev.storj.io/trace/43cf5080575f2b7d
FAIL (2026-05-15 23:20:27 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.905470942s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [............XX....XXX.......................X........X......] duration:
3.875214577s trace:
https://jaeger.dev.storj.io/trace/43cf5080575f2b7d
FAIL (2026-05-15 23:19:56 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.875244474s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [XX.X.X......X....X...XX.X.XX.......XX.X.X..X...........X.X..] duration:
31.720046212s trace:
https://jaeger.dev.storj.io/trace/2c48a3bc0cfb59cd
FAIL (2026-05-15 23:19:02 UTC)
orbiter warning: context deadline exceeded
Duration: 31.720060219s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [.............XX....XXX.......................X........X.....] duration:
3.868880099s trace:
https://jaeger.dev.storj.io/trace/43cf5080575f2b7d
FAIL (2026-05-15 23:19:01 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.868904755s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
19 out of 60 is failed (threshold=3) [XXX.X.X......X....X...XX.X.XX.......XX.X.X..X...........X.X.] duration:
39.312332269s trace:
https://jaeger.dev.storj.io/trace/2c48a3bc0cfb59cd
FAIL (2026-05-15 23:18:30 UTC)
orbiter warning: context deadline exceeded
Duration: 39.312342379s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [..............XX....XXX.......................X........X....] duration:
4.032671806s trace:
https://jaeger.dev.storj.io/trace/43cf5080575f2b7d
FAIL (2026-05-15 23:18:19 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.032691773s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
19 out of 60 is failed (threshold=3) [.XXX.X.X......X....X...XX.X.XX.......XX.X.X..X...........X.X] duration:
1m0.001480777s trace:
https://jaeger.dev.storj.io/trace/2c48a3bc0cfb59cd
FAIL (2026-05-15 23:17:50 UTC)
orbiter warning: context deadline exceeded
Duration: 1m0.001500804s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [...............XX....XXX.......................X........X...] duration:
3.928617491s trace:
https://jaeger.dev.storj.io/trace/43cf5080575f2b7d
FAIL (2026-05-15 23:17:27 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.928628411s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
19 out of 60 is failed (threshold=3) [X.XXX.X.X......X....X...XX.X.XX.......XX.X.X..X...........X.] duration:
28.029105818s trace:
https://jaeger.dev.storj.io/trace/1494a6642cc43016
FAIL (2026-05-15 23:16:23 UTC)
orbiter warning: context deadline exceeded
Duration: 28.029128752s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [................XX....XXX.......................X........X..] duration:
4.124335408s trace:
https://jaeger.dev.storj.io/trace/43cf5080575f2b7d
FAIL (2026-05-15 23:16:17 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.124350737s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
19 out of 60 is failed (threshold=3) [.X.XXX.X.X......X....X...XX.X.XX.......XX.X.X..X...........X] duration:
1m0.001230665s trace:
https://jaeger.dev.storj.io/trace/1494a6642cc43016
FAIL (2026-05-15 23:15:54 UTC)
orbiter warning: context deadline exceeded
Duration: 1m0.001250813s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [.................XX....XXX.......................X........X.] duration:
3.878246823s trace:
https://jaeger.dev.storj.io/trace/43cf5080575f2b7d
FAIL (2026-05-15 23:15:53 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.878259416s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [..................XX....XXX.......................X........X] duration:
5.000702983s trace:
https://jaeger.dev.storj.io/trace/43cf5080575f2b7d
FAIL (2026-05-15 23:15:30 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000719765s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
6 out of 60 is failed (threshold=6) [...................XX....XXX.......................X........] duration:
4.595902754s trace:
https://jaeger.dev.storj.io/trace/30c04283bd323da2
FAIL (2026-05-15 23:14:51 UTC)
orbiter warning: context deadline exceeded
Duration: 4.595931029s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [..X.XXX.X.X......X....X...XX.X.XX.......XX.X.X..X...........] duration:
8.634084843s trace:
https://jaeger.dev.storj.io/trace/7cf4591d633d8777
FAIL (2026-05-15 23:14:08 UTC)
orbiter warning: context deadline exceeded
Duration: 8.634107907s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [X...................XX....XXX.......................X.......] duration:
3.870008803s trace:
https://jaeger.dev.storj.io/trace/30c04283bd323da2
FAIL (2026-05-15 23:13:45 UTC)
orbiter warning: context deadline exceeded
Duration: 3.870087552s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [...X.XXX.X.X......X....X...XX.X.XX.......XX.X.X..X..........] duration:
14.944421171s trace:
https://jaeger.dev.storj.io/trace/7cf4591d633d8777
FAIL (2026-05-15 23:13:07 UTC)
orbiter warning: context deadline exceeded
Duration: 14.944440457s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=6) [XX...................XX....XXX.......................X......] duration:
3.783913003s trace:
https://jaeger.dev.storj.io/trace/30c04283bd323da2
FAIL (2026-05-15 23:12:53 UTC)
orbiter warning: context deadline exceeded
Duration: 3.783994328s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [....X.XXX.X.X......X....X...XX.X.XX.......XX.X.X..X.........] duration:
25.921526634s trace:
https://jaeger.dev.storj.io/trace/7cf4591d633d8777
FAIL (2026-05-15 23:12:37 UTC)
orbiter warning: context deadline exceeded
Duration: 25.921541994s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=6) [.XX...................XX....XXX.......................X.....] duration:
5.000672195s trace:
https://jaeger.dev.storj.io/trace/30c04283bd323da2
FAIL (2026-05-15 23:12:01 UTC)
orbiter warning: context deadline exceeded
Duration: 5.000690148s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [.....X.XXX.X.X......X....X...XX.X.XX.......XX.X.X..X........] duration:
19.767834457s trace:
https://jaeger.dev.storj.io/trace/7cf4591d633d8777
FAIL (2026-05-15 23:11:33 UTC)
orbiter warning: context deadline exceeded
Duration: 19.767848793s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
11.884032749s trace:
https://jaeger.dev.storj.io/trace/15d3b92a22e517da
FAIL (2026-05-15 23:11:29 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload
May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload
May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments...
May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false
May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment...
May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment.
May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment
May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments...
May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces
May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection
May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF
May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF
May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments...
May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil>
Duration: 11.884071051s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [......X.XXX.X.X......X....X...XX.X.XX.......XX.X.X..X.......] duration:
31.915933109s trace:
https://jaeger.dev.storj.io/trace/7cf4591d633d8777
FAIL (2026-05-15 23:11:13 UTC)
orbiter warning: context deadline exceeded
Duration: 31.915981811s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=6) [..XX...................XX....XXX.......................X....] duration:
4.507007224s trace:
https://jaeger.dev.storj.io/trace/30c04283bd323da2
FAIL (2026-05-15 23:11:04 UTC)
orbiter warning: context deadline exceeded
Duration: 4.507033735s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
7.335224661s trace:
https://jaeger.dev.storj.io/trace/15d3b92a22e517da
FAIL (2026-05-15 23:11:02 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload
May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload
May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments...
May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false
May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment...
May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment.
May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment
May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments...
May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces
May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection
May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF
May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF
May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments...
May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil>
Duration: 7.335247915s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=6) [...XX...................XX....XXX.......................X...] duration:
4.724831516s trace:
https://jaeger.dev.storj.io/trace/30c04283bd323da2
FAIL (2026-05-15 23:10:40 UTC)
orbiter warning: context deadline exceeded
Duration: 4.724855863s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [.......X.XXX.X.X......X....X...XX.X.XX.......XX.X.X..X......] duration:
20.837183569s trace:
https://jaeger.dev.storj.io/trace/7cf4591d633d8777
FAIL (2026-05-15 23:09:59 UTC)
orbiter warning: context deadline exceeded
Duration: 20.837207985s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
7.450703848s trace:
https://jaeger.dev.storj.io/trace/15d3b92a22e517da
FAIL (2026-05-15 23:09:48 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload
May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload
May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments...
May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false
May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment...
May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment.
May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment
May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments...
May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces
May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection
May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF
May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF
May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments...
May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil>
Duration: 7.450725831s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=6) [....XX...................XX....XXX.......................X..] duration:
4.656451033s trace:
https://jaeger.dev.storj.io/trace/30c04283bd323da2
FAIL (2026-05-15 23:09:33 UTC)
orbiter warning: context deadline exceeded
Duration: 4.656467684s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
11.952717736s trace:
https://jaeger.dev.storj.io/trace/15d3b92a22e517da
FAIL (2026-05-15 23:09:12 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload
May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload
May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments...
May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false
May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment...
May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment.
May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment
May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments...
May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces
May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection
May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF
May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF
May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments...
May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil>
Duration: 11.95274108s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [........X.XXX.X.X......X....X...XX.X.XX.......XX.X.X..X.....] duration:
16.853544256s trace:
https://jaeger.dev.storj.io/trace/7cf4591d633d8777
FAIL (2026-05-15 23:08:53 UTC)
orbiter warning: context deadline exceeded
Duration: 16.853602175s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
9 out of 60 is failed (threshold=6) [X....XX...................XX....XXX.......................X.] duration:
4.411418409s trace:
https://jaeger.dev.storj.io/trace/30c04283bd323da2
FAIL (2026-05-15 23:08:52 UTC)
orbiter warning: context deadline exceeded
Duration: 4.411441305s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
10.529731873s trace:
https://jaeger.dev.storj.io/trace/15d3b92a22e517da
FAIL (2026-05-15 23:08:26 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload
May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload
May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments...
May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false
May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment...
May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment.
May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment
May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments...
May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces
May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection
May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF
May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF
May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments...
May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil>
Duration: 10.529759347s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [.........X.XXX.X.X......X....X...XX.X.XX.......XX.X.X..X....] duration:
25.469567116s trace:
https://jaeger.dev.storj.io/trace/7cf4591d633d8777
FAIL (2026-05-15 23:08:13 UTC)
orbiter warning: context deadline exceeded
Duration: 25.469582776s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
9 out of 60 is failed (threshold=6) [.X....XX...................XX....XXX.......................X] duration:
5.001375556s trace:
https://jaeger.dev.storj.io/trace/30c04283bd323da2
FAIL (2026-05-15 23:08:12 UTC)
orbiter warning: context deadline exceeded
Duration: 5.001404493s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
10.452072144s trace:
https://jaeger.dev.storj.io/trace/15d3b92a22e517da
FAIL (2026-05-15 23:07:50 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
15 22:28:12.624116703 (363h40m23.144940448s) [upload=348437]: Starting upload
May 15 22:28:12.624173291 (363h40m23.144997036s) [upload=348437]: Done starting upload
May 15 22:28:12.624244949 (363h40m23.145068694s) [upload=348437]: Uploading segments...
May 15 22:28:12.624372873 (363h40m23.145196618s) [upload=348437]: Got next segment. Inline: false
May 15 22:28:12.624400435 (363h40m23.14522418s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 15 22:28:12.624404483 (363h40m23.145228228s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 15 22:28:12.859773855 (363h40m23.3805976s) [seg_pos={0 0}, upload=348437]: Begin upload segment...
May 15 22:28:12.859824852 (363h40m23.380648597s) [seg_pos={0 0}, upload=348437]: Done begin upload segment.
May 15 22:28:12.859850081 (363h40m23.380673826s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.859908192 (363h40m23.380731947s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860421581 (363h40m23.381245326s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860676116 (363h40m23.381499871s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.860844147 (363h40m23.381667882s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861069307 (363h40m23.381893052s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861109183 (363h40m23.381932928s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861359691 (363h40m23.382183436s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861420717 (363h40m23.382244462s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861609167 (363h40m23.382432912s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861840148 (363h40m23.382663893s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.861881827 (363h40m23.382705572s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862114271 (363h40m23.382938016s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862194775 (363h40m23.38301852s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862335683 (363h40m23.383159428s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:12.862560993 (363h40m23.383384738s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Uploading piece...
May 15 22:28:14.637865390 (363h40m25.158689135s) [upload=348437]: Next returned nil segment
May 15 22:28:14.637878394 (363h40m25.158702139s) [upload=348437]: Waiting for error group managing segments...
May 15 22:28:14.934709198 (363h40m25.455532933s) [noise=true , address=77.90.31.19:4841 , piece_id=223YF6YZ4NC2LTDEQE56RVYGCM2SRMT2ILWHCQPKT2FOI7OH5TAQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.938255915 (363h40m25.45907966s) [noise=true , address=145.79.227.38:4977 , piece_id=EHJYZXY6SOX376U4K3YX4H5AROJ7U5BL6LCHJPOG6LAI7EWBEXMQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.952898406 (363h40m25.473722151s) [noise=true , address=145.79.227.39:4014 , piece_id=2Z675YSUNI3WO2Q56TQEG2RT7BCSHTZGV4WFTEN3VHHP4ZUVQF3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.966259342 (363h40m25.487083077s) [noise=true , address=77.90.31.15:4700 , piece_id=BUF6AZ7WDSE2KAW5XAIEILPYWIX4TPLRRBEUNFJZSKLDIYOS6TCA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.978548532 (363h40m25.499372277s) [noise=true , address=145.79.227.13:4803 , piece_id=UXZYNRAIQKYT6I3D4MNU5B6TZ4BY3X6O37N3NCVQDXGQRPFJPZRQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.982278840 (363h40m25.503102585s) [noise=true , address=77.90.31.18:4807 , piece_id=CFEEF7RQYY3QH7E46TP4I3FLU2NAUPQDKDC55427K26LEGZPNTOA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.982420991 (363h40m25.503244736s) [noise=true , address=145.79.227.26:4653 , piece_id=ORN7PNJ5ZIATTIAWFXFOZTWYWAHPDCK6ACPPZKHJWVRL5JSHVGWQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.995745206 (363h40m25.516568951s) [noise=true , address=145.79.227.42:4121 , piece_id=3BZ7JQPMKM7MQDYIU3DHADJZ2VWO7TZ5FRX74DQDHARU3IX3RAOQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:14.996654011 (363h40m25.517477756s) [noise=true , address=77.90.31.11:4558 , piece_id=G3CWJP5C45RKVFLB7FNX2CXAOUAUXLUJDSLFVLXUQEPBXGNSWEBQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.082355686 (363h40m25.603179431s) [noise=true , address=145.79.227.7:4296 , piece_id=HFSKNFFXDEYALYT62BYTLZWV2BXUDZBAQRUJ3F7ARHOD57QEE7RQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.092233315 (363h40m25.61305706s) [noise=true , address=145.79.227.50:4373 , piece_id=Z2DG3MQUGAVDROCN65PQJXFGZEIAOW7KXSH7LXTOGG5YI3MZIZ6A, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.124769251 (363h40m25.645592996s) [noise=true , address=145.79.227.25:4622 , piece_id=5Z6JRT3N4AXV3AFK73H73HKEGQHEXZ5ICIKLS7OFXKKR2SYUWSEA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.596706752 (363h40m26.117530497s) [noise=true , address=45.91.116.4:4947 , piece_id=KNG5P7C2EGWMBFM4TVIULRHSOOQUWOLPAGERVA7ZVXMZDWYB7LYQ, seg_pos={0 0}, upload=348437]: Done uploading piece. err: <nil>
May 15 22:28:15.596727262 (363h40m26.117551007s) [seg_pos={0 0}, upload=348437]: Satisfied cohort requirements after 13 pieces
May 15 22:28:15.596885062 (363h40m26.117708808s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x): protocol: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42910->145.79.227.4:4187: use of closed network connection
May 15 22:28:15.596890423 (363h40m26.117714168s) [noise=true , address=145.79.227.4:4187 , piece_id=HBGVIH4GLFZRDII7RXFZK4I4K32PPA7QPP446LKZTM7V34PWA5LA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597074043 (363h40m26.117897788s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126ZeAKnpjpxa3MgCBU3KZKBDdn8MfZgiixE4zR5fZpPwaQj9NC): protocol: expected piece hash; context canceled; EOF
May 15 22:28:15.597079663 (363h40m26.117903408s) [noise=true , address=145.79.227.15:4878 , piece_id=E7BK3E7KZKYMCS6P57WAUVC54FCHZNDUHLEEIYWLUCV5B2WDYN3Q, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597101896 (363h40m26.117925701s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KYMZAFsJbhWC4CdKRod2Z6qsC6ANcpqmpWSWx7UYajp6JvKH1): protocol: expected piece hash; context canceled; EOF
May 15 22:28:15.597120802 (363h40m26.117944548s) [noise=true , address=145.79.227.8:4334 , piece_id=HHBNK6LESFHRKJDGPXEWWACYVNRXUKURQD5AAKHRLBDWXWXMOWHA, seg_pos={0 0}, upload=348437]: Long tail cancellation due to optimal threshold
May 15 22:28:15.597155839 (363h40m26.117979584s) [seg_pos={0 0}, upload=348437]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 15 22:28:15.597220392 (363h40m26.118044137s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 15 22:28:15.597224660 (363h40m26.118048405s) [upload=348437]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 15 22:28:15.885904640 (363h40m26.406728395s) [upload=348437]: Done uploading segments...
May 15 22:28:15.885925369 (363h40m26.406749114s) [upload=348437]: Upload finished. err: <nil>
Duration: 10.452087715s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [..........X.XXX.X.X......X....X...XX.X.XX.......XX.X.X..X...] duration:
52.53427372s trace:
https://jaeger.dev.storj.io/trace/7cf4591d633d8777
FAIL (2026-05-15 23:07:48 UTC)
orbiter warning: context deadline exceeded
Duration: 52.534286103s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=6) [..X....XX...................XX....XXX.......................] duration:
4.153754916s trace:
https://jaeger.dev.storj.io/trace/434e3e956d043909
FAIL (2026-05-15 23:07:15 UTC)
orbiter warning: context deadline exceeded
Duration: 4.153781476s
core
uplink
us-select-1
us1-placement-20
|