Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 873913 reports (Page 11543 of 17479)
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................]
duration:
5.721896519s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:20:52 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.721935373s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................]
duration:
6.138720777s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:19:41 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 6.138753511s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................]
duration:
6.027906662s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:19:04 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 6.028016332s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................]
duration:
6.301188575s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:18:33 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 6.30121747s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................]
duration:
6.479838229s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:18:16 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 6.479929753s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................]
duration:
5.750641047s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:17:21 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.75067472s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................]
duration:
5.918752959s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:16:54 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.918783979s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................]
duration:
5.823174292s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:15:57 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.823208837s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................]
duration:
6.100312771s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:15:17 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 6.100338259s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................]
duration:
5.57679538s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:14:43 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.576825427s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................]
duration:
5.863863384s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:14:11 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.863890906s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................]
duration:
6.000913546s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:13:24 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 6.000944906s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................]
duration:
6.21141784s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:12:19 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 6.211467015s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................]
duration:
5.64716472s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:12:01 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.647189919s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................]
duration:
5.757797626s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:11:45 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.757816873s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................]
duration:
5.892053023s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:11:30 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.892082129s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................]
duration:
6.012745848s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:11:14 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 6.01277362s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................]
duration:
5.78816457s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:10:49 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.788190249s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................]
duration:
5.960730122s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:10:09 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.960768455s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................]
duration:
6.584899767s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:09:44 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 6.584927299s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............]
duration:
7.701699414s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:09:17 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 7.701723349s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............]
duration:
5.841590395s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:08:19 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.841617718s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............]
duration:
5.953009417s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:07:28 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.95302661s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............]
duration:
5.95857788s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:06:34 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.958659044s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........]
duration:
6.329127819s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:05:32 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 6.329293836s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........]
duration:
5.922616524s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:05:11 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.922644647s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........]
duration:
5.774664383s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:04:06 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.774690253s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........]
duration:
6.518772707s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:03:41 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 6.518799779s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......]
duration:
5.899966109s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:03:10 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.900100213s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......]
duration:
6.095965653s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:02:10 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 6.0959957s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....]
duration:
5.980543646s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:01:44 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.980578041s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....]
duration:
5.691542975s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:01:24 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.691559898s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
5.850270703s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 07:00:23 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.850303767s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
5.7323717s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 06:59:39 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.732402959s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration:
5.933740434s
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 06:58:51 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 5.933783266s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X]
duration:
458.685033ms
trace:
https://jaeger.dev.storj.io/trace/4d3274db26fab217
FAIL (2025-12-12 06:57:55 UTC)
orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": read tcp 188.166.225.170:59068->45.38.24.2:443: read: connection reset by peer
Duration: 458.711324ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................]
duration:
8.312424284s
trace:
https://jaeger.dev.storj.io/trace/1feedb9fc9b15c7f
FAIL (2025-12-12 06:38:51 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, Piece ID: AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ) context deadline exceeded; piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q) context deadline exceeded; piecestore: (Node ID: 12HZjKTnjpeNU5B6muVHGrmWQS8GsyuYTW6cLxzNk5CGt3b8PJS, Piece ID: T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A) context deadline exceeded; piecestore: (Node ID: 1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, Piece ID: V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA) context deadline exceeded
======LOGS======
Dec 12 05:52:28.758186788 (158h23m28.629269833s) [upload=138545]: Starting upload
Dec 12 05:52:28.758268054 (158h23m28.629351089s) [upload=138545]: Done starting upload
Dec 12 05:52:28.758478524 (158h23m28.62956157s) [upload=138545]: Uploading segments...
Dec 12 05:52:28.758589055 (158h23m28.62967209s) [upload=138545]: Got next segment. Inline: false
Dec 12 05:52:28.758627448 (158h23m28.629710493s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 12 05:52:28.758633229 (158h23m28.629716264s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 12 05:52:29.284642531 (158h23m29.155725586s) [seg_pos={0 0}, upload=138545]: Begin upload segment...
Dec 12 05:52:29.284698698 (158h23m29.155781743s) [seg_pos={0 0}, upload=138545]: Done begin upload segment.
Dec 12 05:52:29.284747200 (158h23m29.155830245s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.285714762 (158h23m29.156797808s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286167084 (158h23m29.157250119s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286680902 (158h23m29.157763937s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.287180654 (158h23m29.158263699s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288071100 (158h23m29.159154145s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288112027 (158h23m29.159195083s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289064060 (158h23m29.160147095s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289119456 (158h23m29.160202501s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290114690 (158h23m29.161197735s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290153143 (158h23m29.161236189s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291232188 (158h23m29.162315233s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291345174 (158h23m29.162428219s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291791403 (158h23m29.162874458s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292096845 (158h23m29.16317988s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292469154 (158h23m29.163552199s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:31.055397931 (158h23m30.926480977s) [upload=138545]: Next returned nil segment
Dec 12 05:52:31.055414492 (158h23m30.926497538s) [upload=138545]: Waiting for error group managing segments...
Dec 12 05:52:31.488724096 (158h23m31.359807131s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.526825088 (158h23m31.397908123s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.568810436 (158h23m31.439893481s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.593069069 (158h23m31.464152114s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.684591374 (158h23m31.555674419s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.981164324 (158h23m31.852247369s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.086675528 (158h23m31.957758573s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.328097079 (158h23m32.199180124s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.363427452 (158h23m33.234510497s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.581309902 (158h23m33.452392947s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.597965646 (158h23m33.469048692s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.127692344 (158h23m34.998775389s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520638237 (158h23m35.391721292s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520664637 (158h23m35.391747712s) [seg_pos={0 0}, upload=138545]: Segment reached optimal threshold of 13 pieces.
Dec 12 05:52:35.521193143 (158h23m35.392276239s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: read tcp 188.166.225.170:56866->145.79.227.25:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521205126 (158h23m35.392288181s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.521775052 (158h23m35.392858107s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA): protocol: read tcp 188.166.225.170:35500->145.79.227.43:4159: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521788437 (158h23m35.392871472s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522125940 (158h23m35.393208985s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17x7QtnZsENMZPU3tqkh8jEd29JdowLvT855W4kuourgUA6y5C): protocol: expected piece hash; context canceled; EOF
Dec 12 05:52:35.522137161 (158h23m35.393220206s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522160766 (158h23m35.393243801s) [seg_pos={0 0}, upload=138545]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 12 05:52:35.522242732 (158h23m35.393325777s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 12 05:52:35.522248944 (158h23m35.393331989s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 12 05:52:35.844205119 (158h23m35.715288164s) [upload=138545]: Done uploading segments...
Dec 12 05:52:35.844231129 (158h23m35.715314164s) [upload=138545]: Upload finished. err: <nil>
Duration: 8.312502845s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................]
duration:
7.22550529s
trace:
https://jaeger.dev.storj.io/trace/1feedb9fc9b15c7f
FAIL (2025-12-12 06:37:53 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, Piece ID: AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ) context deadline exceeded; piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q) context deadline exceeded; piecestore: (Node ID: 12HZjKTnjpeNU5B6muVHGrmWQS8GsyuYTW6cLxzNk5CGt3b8PJS, Piece ID: T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A) context deadline exceeded; piecestore: (Node ID: 1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, Piece ID: V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA) context deadline exceeded
======LOGS======
Dec 12 05:52:28.758186788 (158h23m28.629269833s) [upload=138545]: Starting upload
Dec 12 05:52:28.758268054 (158h23m28.629351089s) [upload=138545]: Done starting upload
Dec 12 05:52:28.758478524 (158h23m28.62956157s) [upload=138545]: Uploading segments...
Dec 12 05:52:28.758589055 (158h23m28.62967209s) [upload=138545]: Got next segment. Inline: false
Dec 12 05:52:28.758627448 (158h23m28.629710493s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 12 05:52:28.758633229 (158h23m28.629716264s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 12 05:52:29.284642531 (158h23m29.155725586s) [seg_pos={0 0}, upload=138545]: Begin upload segment...
Dec 12 05:52:29.284698698 (158h23m29.155781743s) [seg_pos={0 0}, upload=138545]: Done begin upload segment.
Dec 12 05:52:29.284747200 (158h23m29.155830245s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.285714762 (158h23m29.156797808s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286167084 (158h23m29.157250119s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286680902 (158h23m29.157763937s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.287180654 (158h23m29.158263699s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288071100 (158h23m29.159154145s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288112027 (158h23m29.159195083s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289064060 (158h23m29.160147095s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289119456 (158h23m29.160202501s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290114690 (158h23m29.161197735s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290153143 (158h23m29.161236189s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291232188 (158h23m29.162315233s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291345174 (158h23m29.162428219s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291791403 (158h23m29.162874458s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292096845 (158h23m29.16317988s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292469154 (158h23m29.163552199s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:31.055397931 (158h23m30.926480977s) [upload=138545]: Next returned nil segment
Dec 12 05:52:31.055414492 (158h23m30.926497538s) [upload=138545]: Waiting for error group managing segments...
Dec 12 05:52:31.488724096 (158h23m31.359807131s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.526825088 (158h23m31.397908123s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.568810436 (158h23m31.439893481s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.593069069 (158h23m31.464152114s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.684591374 (158h23m31.555674419s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.981164324 (158h23m31.852247369s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.086675528 (158h23m31.957758573s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.328097079 (158h23m32.199180124s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.363427452 (158h23m33.234510497s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.581309902 (158h23m33.452392947s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.597965646 (158h23m33.469048692s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.127692344 (158h23m34.998775389s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520638237 (158h23m35.391721292s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520664637 (158h23m35.391747712s) [seg_pos={0 0}, upload=138545]: Segment reached optimal threshold of 13 pieces.
Dec 12 05:52:35.521193143 (158h23m35.392276239s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: read tcp 188.166.225.170:56866->145.79.227.25:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521205126 (158h23m35.392288181s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.521775052 (158h23m35.392858107s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA): protocol: read tcp 188.166.225.170:35500->145.79.227.43:4159: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521788437 (158h23m35.392871472s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522125940 (158h23m35.393208985s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17x7QtnZsENMZPU3tqkh8jEd29JdowLvT855W4kuourgUA6y5C): protocol: expected piece hash; context canceled; EOF
Dec 12 05:52:35.522137161 (158h23m35.393220206s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522160766 (158h23m35.393243801s) [seg_pos={0 0}, upload=138545]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 12 05:52:35.522242732 (158h23m35.393325777s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 12 05:52:35.522248944 (158h23m35.393331989s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 12 05:52:35.844205119 (158h23m35.715288164s) [upload=138545]: Done uploading segments...
Dec 12 05:52:35.844231129 (158h23m35.715314164s) [upload=138545]: Upload finished. err: <nil>
Duration: 7.22553642s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................]
duration:
7.093845058s
trace:
https://jaeger.dev.storj.io/trace/1feedb9fc9b15c7f
FAIL (2025-12-12 06:37:24 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, Piece ID: AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ) context deadline exceeded; piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q) context deadline exceeded; piecestore: (Node ID: 12HZjKTnjpeNU5B6muVHGrmWQS8GsyuYTW6cLxzNk5CGt3b8PJS, Piece ID: T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A) context deadline exceeded; piecestore: (Node ID: 1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, Piece ID: V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA) context deadline exceeded
======LOGS======
Dec 12 05:52:28.758186788 (158h23m28.629269833s) [upload=138545]: Starting upload
Dec 12 05:52:28.758268054 (158h23m28.629351089s) [upload=138545]: Done starting upload
Dec 12 05:52:28.758478524 (158h23m28.62956157s) [upload=138545]: Uploading segments...
Dec 12 05:52:28.758589055 (158h23m28.62967209s) [upload=138545]: Got next segment. Inline: false
Dec 12 05:52:28.758627448 (158h23m28.629710493s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 12 05:52:28.758633229 (158h23m28.629716264s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 12 05:52:29.284642531 (158h23m29.155725586s) [seg_pos={0 0}, upload=138545]: Begin upload segment...
Dec 12 05:52:29.284698698 (158h23m29.155781743s) [seg_pos={0 0}, upload=138545]: Done begin upload segment.
Dec 12 05:52:29.284747200 (158h23m29.155830245s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.285714762 (158h23m29.156797808s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286167084 (158h23m29.157250119s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286680902 (158h23m29.157763937s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.287180654 (158h23m29.158263699s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288071100 (158h23m29.159154145s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288112027 (158h23m29.159195083s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289064060 (158h23m29.160147095s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289119456 (158h23m29.160202501s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290114690 (158h23m29.161197735s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290153143 (158h23m29.161236189s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291232188 (158h23m29.162315233s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291345174 (158h23m29.162428219s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291791403 (158h23m29.162874458s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292096845 (158h23m29.16317988s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292469154 (158h23m29.163552199s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:31.055397931 (158h23m30.926480977s) [upload=138545]: Next returned nil segment
Dec 12 05:52:31.055414492 (158h23m30.926497538s) [upload=138545]: Waiting for error group managing segments...
Dec 12 05:52:31.488724096 (158h23m31.359807131s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.526825088 (158h23m31.397908123s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.568810436 (158h23m31.439893481s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.593069069 (158h23m31.464152114s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.684591374 (158h23m31.555674419s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.981164324 (158h23m31.852247369s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.086675528 (158h23m31.957758573s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.328097079 (158h23m32.199180124s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.363427452 (158h23m33.234510497s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.581309902 (158h23m33.452392947s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.597965646 (158h23m33.469048692s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.127692344 (158h23m34.998775389s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520638237 (158h23m35.391721292s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520664637 (158h23m35.391747712s) [seg_pos={0 0}, upload=138545]: Segment reached optimal threshold of 13 pieces.
Dec 12 05:52:35.521193143 (158h23m35.392276239s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: read tcp 188.166.225.170:56866->145.79.227.25:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521205126 (158h23m35.392288181s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.521775052 (158h23m35.392858107s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA): protocol: read tcp 188.166.225.170:35500->145.79.227.43:4159: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521788437 (158h23m35.392871472s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522125940 (158h23m35.393208985s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17x7QtnZsENMZPU3tqkh8jEd29JdowLvT855W4kuourgUA6y5C): protocol: expected piece hash; context canceled; EOF
Dec 12 05:52:35.522137161 (158h23m35.393220206s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522160766 (158h23m35.393243801s) [seg_pos={0 0}, upload=138545]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 12 05:52:35.522242732 (158h23m35.393325777s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 12 05:52:35.522248944 (158h23m35.393331989s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 12 05:52:35.844205119 (158h23m35.715288164s) [upload=138545]: Done uploading segments...
Dec 12 05:52:35.844231129 (158h23m35.715314164s) [upload=138545]: Upload finished. err: <nil>
Duration: 7.093881257s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................]
duration:
6.02460259s
trace:
https://jaeger.dev.storj.io/trace/76c0b0a8c78b2475
FAIL (2025-12-12 06:36:30 UTC)
orbiter warning: operation: ListObjects
uplink: metainfo: metaclient: context deadline exceeded
Duration: 6.024634841s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................]
duration:
6.72194913s
trace:
https://jaeger.dev.storj.io/trace/1feedb9fc9b15c7f
FAIL (2025-12-12 06:36:18 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, Piece ID: AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ) context deadline exceeded; piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q) context deadline exceeded; piecestore: (Node ID: 12HZjKTnjpeNU5B6muVHGrmWQS8GsyuYTW6cLxzNk5CGt3b8PJS, Piece ID: T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A) context deadline exceeded; piecestore: (Node ID: 1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, Piece ID: V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA) context deadline exceeded
======LOGS======
Dec 12 05:52:28.758186788 (158h23m28.629269833s) [upload=138545]: Starting upload
Dec 12 05:52:28.758268054 (158h23m28.629351089s) [upload=138545]: Done starting upload
Dec 12 05:52:28.758478524 (158h23m28.62956157s) [upload=138545]: Uploading segments...
Dec 12 05:52:28.758589055 (158h23m28.62967209s) [upload=138545]: Got next segment. Inline: false
Dec 12 05:52:28.758627448 (158h23m28.629710493s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 12 05:52:28.758633229 (158h23m28.629716264s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 12 05:52:29.284642531 (158h23m29.155725586s) [seg_pos={0 0}, upload=138545]: Begin upload segment...
Dec 12 05:52:29.284698698 (158h23m29.155781743s) [seg_pos={0 0}, upload=138545]: Done begin upload segment.
Dec 12 05:52:29.284747200 (158h23m29.155830245s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.285714762 (158h23m29.156797808s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286167084 (158h23m29.157250119s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286680902 (158h23m29.157763937s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.287180654 (158h23m29.158263699s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288071100 (158h23m29.159154145s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288112027 (158h23m29.159195083s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289064060 (158h23m29.160147095s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289119456 (158h23m29.160202501s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290114690 (158h23m29.161197735s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290153143 (158h23m29.161236189s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291232188 (158h23m29.162315233s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291345174 (158h23m29.162428219s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291791403 (158h23m29.162874458s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292096845 (158h23m29.16317988s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292469154 (158h23m29.163552199s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:31.055397931 (158h23m30.926480977s) [upload=138545]: Next returned nil segment
Dec 12 05:52:31.055414492 (158h23m30.926497538s) [upload=138545]: Waiting for error group managing segments...
Dec 12 05:52:31.488724096 (158h23m31.359807131s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.526825088 (158h23m31.397908123s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.568810436 (158h23m31.439893481s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.593069069 (158h23m31.464152114s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.684591374 (158h23m31.555674419s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.981164324 (158h23m31.852247369s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.086675528 (158h23m31.957758573s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.328097079 (158h23m32.199180124s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.363427452 (158h23m33.234510497s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.581309902 (158h23m33.452392947s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.597965646 (158h23m33.469048692s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.127692344 (158h23m34.998775389s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520638237 (158h23m35.391721292s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520664637 (158h23m35.391747712s) [seg_pos={0 0}, upload=138545]: Segment reached optimal threshold of 13 pieces.
Dec 12 05:52:35.521193143 (158h23m35.392276239s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: read tcp 188.166.225.170:56866->145.79.227.25:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521205126 (158h23m35.392288181s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.521775052 (158h23m35.392858107s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA): protocol: read tcp 188.166.225.170:35500->145.79.227.43:4159: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521788437 (158h23m35.392871472s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522125940 (158h23m35.393208985s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17x7QtnZsENMZPU3tqkh8jEd29JdowLvT855W4kuourgUA6y5C): protocol: expected piece hash; context canceled; EOF
Dec 12 05:52:35.522137161 (158h23m35.393220206s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522160766 (158h23m35.393243801s) [seg_pos={0 0}, upload=138545]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 12 05:52:35.522242732 (158h23m35.393325777s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 12 05:52:35.522248944 (158h23m35.393331989s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 12 05:52:35.844205119 (158h23m35.715288164s) [upload=138545]: Done uploading segments...
Dec 12 05:52:35.844231129 (158h23m35.715314164s) [upload=138545]: Upload finished. err: <nil>
Duration: 6.721977424s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................]
duration:
6.185653604s
trace:
https://jaeger.dev.storj.io/trace/76c0b0a8c78b2475
FAIL (2025-12-12 06:35:48 UTC)
orbiter warning: operation: ListObjects
uplink: metainfo: metaclient: context deadline exceeded
Duration: 6.185718438s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................]
duration:
7.119162701s
trace:
https://jaeger.dev.storj.io/trace/1feedb9fc9b15c7f
FAIL (2025-12-12 06:35:35 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, Piece ID: AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ) context deadline exceeded; piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q) context deadline exceeded; piecestore: (Node ID: 12HZjKTnjpeNU5B6muVHGrmWQS8GsyuYTW6cLxzNk5CGt3b8PJS, Piece ID: T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A) context deadline exceeded; piecestore: (Node ID: 1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, Piece ID: V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA) context deadline exceeded
======LOGS======
Dec 12 05:52:28.758186788 (158h23m28.629269833s) [upload=138545]: Starting upload
Dec 12 05:52:28.758268054 (158h23m28.629351089s) [upload=138545]: Done starting upload
Dec 12 05:52:28.758478524 (158h23m28.62956157s) [upload=138545]: Uploading segments...
Dec 12 05:52:28.758589055 (158h23m28.62967209s) [upload=138545]: Got next segment. Inline: false
Dec 12 05:52:28.758627448 (158h23m28.629710493s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 12 05:52:28.758633229 (158h23m28.629716264s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 12 05:52:29.284642531 (158h23m29.155725586s) [seg_pos={0 0}, upload=138545]: Begin upload segment...
Dec 12 05:52:29.284698698 (158h23m29.155781743s) [seg_pos={0 0}, upload=138545]: Done begin upload segment.
Dec 12 05:52:29.284747200 (158h23m29.155830245s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.285714762 (158h23m29.156797808s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286167084 (158h23m29.157250119s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286680902 (158h23m29.157763937s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.287180654 (158h23m29.158263699s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288071100 (158h23m29.159154145s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288112027 (158h23m29.159195083s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289064060 (158h23m29.160147095s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289119456 (158h23m29.160202501s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290114690 (158h23m29.161197735s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290153143 (158h23m29.161236189s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291232188 (158h23m29.162315233s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291345174 (158h23m29.162428219s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291791403 (158h23m29.162874458s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292096845 (158h23m29.16317988s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292469154 (158h23m29.163552199s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:31.055397931 (158h23m30.926480977s) [upload=138545]: Next returned nil segment
Dec 12 05:52:31.055414492 (158h23m30.926497538s) [upload=138545]: Waiting for error group managing segments...
Dec 12 05:52:31.488724096 (158h23m31.359807131s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.526825088 (158h23m31.397908123s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.568810436 (158h23m31.439893481s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.593069069 (158h23m31.464152114s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.684591374 (158h23m31.555674419s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.981164324 (158h23m31.852247369s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.086675528 (158h23m31.957758573s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.328097079 (158h23m32.199180124s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.363427452 (158h23m33.234510497s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.581309902 (158h23m33.452392947s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.597965646 (158h23m33.469048692s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.127692344 (158h23m34.998775389s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520638237 (158h23m35.391721292s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520664637 (158h23m35.391747712s) [seg_pos={0 0}, upload=138545]: Segment reached optimal threshold of 13 pieces.
Dec 12 05:52:35.521193143 (158h23m35.392276239s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: read tcp 188.166.225.170:56866->145.79.227.25:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521205126 (158h23m35.392288181s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.521775052 (158h23m35.392858107s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA): protocol: read tcp 188.166.225.170:35500->145.79.227.43:4159: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521788437 (158h23m35.392871472s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522125940 (158h23m35.393208985s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17x7QtnZsENMZPU3tqkh8jEd29JdowLvT855W4kuourgUA6y5C): protocol: expected piece hash; context canceled; EOF
Dec 12 05:52:35.522137161 (158h23m35.393220206s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522160766 (158h23m35.393243801s) [seg_pos={0 0}, upload=138545]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 12 05:52:35.522242732 (158h23m35.393325777s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 12 05:52:35.522248944 (158h23m35.393331989s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 12 05:52:35.844205119 (158h23m35.715288164s) [upload=138545]: Done uploading segments...
Dec 12 05:52:35.844231129 (158h23m35.715314164s) [upload=138545]: Upload finished. err: <nil>
Duration: 7.11918806s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................]
duration:
5.781052386s
trace:
https://jaeger.dev.storj.io/trace/76c0b0a8c78b2475
FAIL (2025-12-12 06:35:11 UTC)
orbiter warning: operation: ListObjects
uplink: metainfo: metaclient: context deadline exceeded
Duration: 5.781087782s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................]
duration:
6.903794396s
trace:
https://jaeger.dev.storj.io/trace/1feedb9fc9b15c7f
FAIL (2025-12-12 06:35:04 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, Piece ID: AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ) context deadline exceeded; piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q) context deadline exceeded; piecestore: (Node ID: 12HZjKTnjpeNU5B6muVHGrmWQS8GsyuYTW6cLxzNk5CGt3b8PJS, Piece ID: T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A) context deadline exceeded; piecestore: (Node ID: 1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, Piece ID: V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA) context deadline exceeded
======LOGS======
Dec 12 05:52:28.758186788 (158h23m28.629269833s) [upload=138545]: Starting upload
Dec 12 05:52:28.758268054 (158h23m28.629351089s) [upload=138545]: Done starting upload
Dec 12 05:52:28.758478524 (158h23m28.62956157s) [upload=138545]: Uploading segments...
Dec 12 05:52:28.758589055 (158h23m28.62967209s) [upload=138545]: Got next segment. Inline: false
Dec 12 05:52:28.758627448 (158h23m28.629710493s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 12 05:52:28.758633229 (158h23m28.629716264s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 12 05:52:29.284642531 (158h23m29.155725586s) [seg_pos={0 0}, upload=138545]: Begin upload segment...
Dec 12 05:52:29.284698698 (158h23m29.155781743s) [seg_pos={0 0}, upload=138545]: Done begin upload segment.
Dec 12 05:52:29.284747200 (158h23m29.155830245s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.285714762 (158h23m29.156797808s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286167084 (158h23m29.157250119s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286680902 (158h23m29.157763937s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.287180654 (158h23m29.158263699s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288071100 (158h23m29.159154145s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288112027 (158h23m29.159195083s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289064060 (158h23m29.160147095s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289119456 (158h23m29.160202501s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290114690 (158h23m29.161197735s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290153143 (158h23m29.161236189s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291232188 (158h23m29.162315233s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291345174 (158h23m29.162428219s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291791403 (158h23m29.162874458s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292096845 (158h23m29.16317988s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292469154 (158h23m29.163552199s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:31.055397931 (158h23m30.926480977s) [upload=138545]: Next returned nil segment
Dec 12 05:52:31.055414492 (158h23m30.926497538s) [upload=138545]: Waiting for error group managing segments...
Dec 12 05:52:31.488724096 (158h23m31.359807131s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.526825088 (158h23m31.397908123s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.568810436 (158h23m31.439893481s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.593069069 (158h23m31.464152114s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.684591374 (158h23m31.555674419s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.981164324 (158h23m31.852247369s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.086675528 (158h23m31.957758573s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.328097079 (158h23m32.199180124s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.363427452 (158h23m33.234510497s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.581309902 (158h23m33.452392947s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.597965646 (158h23m33.469048692s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.127692344 (158h23m34.998775389s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520638237 (158h23m35.391721292s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520664637 (158h23m35.391747712s) [seg_pos={0 0}, upload=138545]: Segment reached optimal threshold of 13 pieces.
Dec 12 05:52:35.521193143 (158h23m35.392276239s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: read tcp 188.166.225.170:56866->145.79.227.25:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521205126 (158h23m35.392288181s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.521775052 (158h23m35.392858107s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA): protocol: read tcp 188.166.225.170:35500->145.79.227.43:4159: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521788437 (158h23m35.392871472s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522125940 (158h23m35.393208985s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17x7QtnZsENMZPU3tqkh8jEd29JdowLvT855W4kuourgUA6y5C): protocol: expected piece hash; context canceled; EOF
Dec 12 05:52:35.522137161 (158h23m35.393220206s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522160766 (158h23m35.393243801s) [seg_pos={0 0}, upload=138545]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 12 05:52:35.522242732 (158h23m35.393325777s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 12 05:52:35.522248944 (158h23m35.393331989s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 12 05:52:35.844205119 (158h23m35.715288164s) [upload=138545]: Done uploading segments...
Dec 12 05:52:35.844231129 (158h23m35.715314164s) [upload=138545]: Upload finished. err: <nil>
Duration: 6.903829423s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................]
duration:
6.327965874s
trace:
https://jaeger.dev.storj.io/trace/1feedb9fc9b15c7f
FAIL (2025-12-12 06:34:35 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, Piece ID: AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ) context deadline exceeded; piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q) context deadline exceeded; piecestore: (Node ID: 12HZjKTnjpeNU5B6muVHGrmWQS8GsyuYTW6cLxzNk5CGt3b8PJS, Piece ID: T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A) context deadline exceeded; piecestore: (Node ID: 1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, Piece ID: V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA) context deadline exceeded
======LOGS======
Dec 12 05:52:28.758186788 (158h23m28.629269833s) [upload=138545]: Starting upload
Dec 12 05:52:28.758268054 (158h23m28.629351089s) [upload=138545]: Done starting upload
Dec 12 05:52:28.758478524 (158h23m28.62956157s) [upload=138545]: Uploading segments...
Dec 12 05:52:28.758589055 (158h23m28.62967209s) [upload=138545]: Got next segment. Inline: false
Dec 12 05:52:28.758627448 (158h23m28.629710493s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 12 05:52:28.758633229 (158h23m28.629716264s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 12 05:52:29.284642531 (158h23m29.155725586s) [seg_pos={0 0}, upload=138545]: Begin upload segment...
Dec 12 05:52:29.284698698 (158h23m29.155781743s) [seg_pos={0 0}, upload=138545]: Done begin upload segment.
Dec 12 05:52:29.284747200 (158h23m29.155830245s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.285714762 (158h23m29.156797808s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286167084 (158h23m29.157250119s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286680902 (158h23m29.157763937s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.287180654 (158h23m29.158263699s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288071100 (158h23m29.159154145s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288112027 (158h23m29.159195083s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289064060 (158h23m29.160147095s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289119456 (158h23m29.160202501s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290114690 (158h23m29.161197735s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290153143 (158h23m29.161236189s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291232188 (158h23m29.162315233s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291345174 (158h23m29.162428219s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291791403 (158h23m29.162874458s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292096845 (158h23m29.16317988s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292469154 (158h23m29.163552199s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:31.055397931 (158h23m30.926480977s) [upload=138545]: Next returned nil segment
Dec 12 05:52:31.055414492 (158h23m30.926497538s) [upload=138545]: Waiting for error group managing segments...
Dec 12 05:52:31.488724096 (158h23m31.359807131s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.526825088 (158h23m31.397908123s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.568810436 (158h23m31.439893481s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.593069069 (158h23m31.464152114s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.684591374 (158h23m31.555674419s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.981164324 (158h23m31.852247369s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.086675528 (158h23m31.957758573s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.328097079 (158h23m32.199180124s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.363427452 (158h23m33.234510497s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.581309902 (158h23m33.452392947s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.597965646 (158h23m33.469048692s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.127692344 (158h23m34.998775389s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520638237 (158h23m35.391721292s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520664637 (158h23m35.391747712s) [seg_pos={0 0}, upload=138545]: Segment reached optimal threshold of 13 pieces.
Dec 12 05:52:35.521193143 (158h23m35.392276239s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: read tcp 188.166.225.170:56866->145.79.227.25:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521205126 (158h23m35.392288181s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.521775052 (158h23m35.392858107s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA): protocol: read tcp 188.166.225.170:35500->145.79.227.43:4159: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521788437 (158h23m35.392871472s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522125940 (158h23m35.393208985s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17x7QtnZsENMZPU3tqkh8jEd29JdowLvT855W4kuourgUA6y5C): protocol: expected piece hash; context canceled; EOF
Dec 12 05:52:35.522137161 (158h23m35.393220206s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522160766 (158h23m35.393243801s) [seg_pos={0 0}, upload=138545]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 12 05:52:35.522242732 (158h23m35.393325777s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 12 05:52:35.522248944 (158h23m35.393331989s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 12 05:52:35.844205119 (158h23m35.715288164s) [upload=138545]: Done uploading segments...
Dec 12 05:52:35.844231129 (158h23m35.715314164s) [upload=138545]: Upload finished. err: <nil>
Duration: 6.32799555s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................]
duration:
5.942871798s
trace:
https://jaeger.dev.storj.io/trace/76c0b0a8c78b2475
FAIL (2025-12-12 06:34:24 UTC)
orbiter warning: operation: ListObjects
uplink: metainfo: metaclient: context deadline exceeded
Duration: 5.942900303s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................]
duration:
6.392323992s
trace:
https://jaeger.dev.storj.io/trace/1feedb9fc9b15c7f
FAIL (2025-12-12 06:33:26 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, Piece ID: AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ) context deadline exceeded; piecestore: (Node ID: 12HWX3mC14qJg7WdCetaBtS2GcGQFhzSaqUbKrYFBHZKdyFAAgX, Piece ID: ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q) context deadline exceeded; piecestore: (Node ID: 12HZjKTnjpeNU5B6muVHGrmWQS8GsyuYTW6cLxzNk5CGt3b8PJS, Piece ID: T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A) context deadline exceeded; piecestore: (Node ID: 1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, Piece ID: V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA) context deadline exceeded
======LOGS======
Dec 12 05:52:28.758186788 (158h23m28.629269833s) [upload=138545]: Starting upload
Dec 12 05:52:28.758268054 (158h23m28.629351089s) [upload=138545]: Done starting upload
Dec 12 05:52:28.758478524 (158h23m28.62956157s) [upload=138545]: Uploading segments...
Dec 12 05:52:28.758589055 (158h23m28.62967209s) [upload=138545]: Got next segment. Inline: false
Dec 12 05:52:28.758627448 (158h23m28.629710493s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 12 05:52:28.758633229 (158h23m28.629716264s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 12 05:52:29.284642531 (158h23m29.155725586s) [seg_pos={0 0}, upload=138545]: Begin upload segment...
Dec 12 05:52:29.284698698 (158h23m29.155781743s) [seg_pos={0 0}, upload=138545]: Done begin upload segment.
Dec 12 05:52:29.284747200 (158h23m29.155830245s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.285714762 (158h23m29.156797808s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286167084 (158h23m29.157250119s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.286680902 (158h23m29.157763937s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.287180654 (158h23m29.158263699s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288071100 (158h23m29.159154145s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.288112027 (158h23m29.159195083s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289064060 (158h23m29.160147095s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.289119456 (158h23m29.160202501s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290114690 (158h23m29.161197735s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.290153143 (158h23m29.161236189s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291232188 (158h23m29.162315233s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291345174 (158h23m29.162428219s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.291791403 (158h23m29.162874458s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292096845 (158h23m29.16317988s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:29.292469154 (158h23m29.163552199s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Uploading piece...
Dec 12 05:52:31.055397931 (158h23m30.926480977s) [upload=138545]: Next returned nil segment
Dec 12 05:52:31.055414492 (158h23m30.926497538s) [upload=138545]: Waiting for error group managing segments...
Dec 12 05:52:31.488724096 (158h23m31.359807131s) [noise=true , address=77.90.31.18:4805     , piece_id=256JZ54CU4FB2R7TGIAMUUWWLZYHQKFB3HYEU7IXB2L34XBCN5FA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.526825088 (158h23m31.397908123s) [noise=true , address=77.90.31.16:4735     , piece_id=Q55QMXTDMSRAIU3NVRA77P3WXRUGUVKPUFSLOP4B5MU67S7OUR2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.568810436 (158h23m31.439893481s) [noise=true , address=145.79.227.36:4517   , piece_id=T2O7DFZXJ6TEFMX75K5JK7X5PJR5GGPY65DHJDKYRZIHGZI46W2A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.593069069 (158h23m31.464152114s) [noise=true , address=145.79.227.34:4905   , piece_id=OCJKQDITIBYRSSUTSVWJYCOVEKMMHT5FBPFIK3ZPITWHN6O6HN5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.684591374 (158h23m31.555674419s) [noise=true , address=145.79.227.49:4337   , piece_id=67URJKLDGXQEMUL4R5FFY4UYLUPQCTIAV2IUDOYMCSAPDIMGYA7A, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:31.981164324 (158h23m31.852247369s) [noise=true , address=145.79.227.19:4078   , piece_id=XXQDBSEQC2JH2UDIPGIUL35OOONZTK6TACRVTQRNR4X5TADMFJOQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.086675528 (158h23m31.957758573s) [noise=true , address=145.79.227.27:4689   , piece_id=PSZIMT7PXTRLC6QQSHMR5IO2WQQBBB62DNZPQQCB5GCM2GYG76PA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:32.328097079 (158h23m32.199180124s) [noise=true , address=145.79.227.52:4447   , piece_id=AGN7AA5B5BVFSTKCS3TN75K2AO7RANBXMJFFLLUGFLBC5A54NNBQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.363427452 (158h23m33.234510497s) [noise=true , address=145.79.227.47:4266   , piece_id=V6FYPFYCCQTS3GULHYFZOY6JDAVKHNMQ4I77C6KFWSFYZRGBUASA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.581309902 (158h23m33.452392947s) [noise=true , address=145.79.227.46:4601   , piece_id=6F2OKCLBRK3JRME7NOEUJ4TEIVIOX3CJGKCVGROCHVZ6GDW3KYPA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:33.597965646 (158h23m33.469048692s) [noise=true , address=145.79.227.32:4836   , piece_id=IZWT5CMWWQFF2MVLK7RWQ3TJ7SGMILNNRTYOB6OMNQ5VWVT42KDQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.127692344 (158h23m34.998775389s) [noise=true , address=145.79.227.51:4410   , piece_id=ZE4XSENLNTPSLNUHRBCSFIKEB3FDZCVMD3TLJJSQDG6AGASJAW5Q, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520638237 (158h23m35.391721292s) [noise=true , address=145.79.227.4:4185    , piece_id=HKTBVHTFWTNTAVGRNZTFQRZZHNBSSWNHEXEXWGUBZAWUM2OHTRLQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: <nil>
Dec 12 05:52:35.520664637 (158h23m35.391747712s) [seg_pos={0 0}, upload=138545]: Segment reached optimal threshold of 13 pieces.
Dec 12 05:52:35.521193143 (158h23m35.392276239s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: read tcp 188.166.225.170:56866->145.79.227.25:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521205126 (158h23m35.392288181s) [noise=true , address=145.79.227.25:4618   , piece_id=M47MFLF7ZOWMCGD7K7WIHA6I3RFLI4MI7TSUWNH7WMTEYD2AC7ZQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.521775052 (158h23m35.392858107s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ULUPKjJwZocjV1R6Dy7vmWSrZ4hKx3YAHLAihgtMKf8Uw2UgA): protocol: read tcp 188.166.225.170:35500->145.79.227.43:4159: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 12 05:52:35.521788437 (158h23m35.392871472s) [noise=true , address=145.79.227.43:4159   , piece_id=WNTFV3DGNXSBDQ74W6WKY7S2BPFRLMXT5LOC2DMJIR6ZOPCPTKQA, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522125940 (158h23m35.393208985s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17x7QtnZsENMZPU3tqkh8jEd29JdowLvT855W4kuourgUA6y5C): protocol: expected piece hash; context canceled; EOF
Dec 12 05:52:35.522137161 (158h23m35.393220206s) [noise=true , address=145.79.227.41:4085   , piece_id=2NAFYOEP5YOQRA6LJGOX24EXNNGIL2NPR6WKETE5WJWVSILDHJJQ, seg_pos={0 0}, upload=138545]: Long tail cancellation due to optimal threshold
Dec 12 05:52:35.522160766 (158h23m35.393243801s) [seg_pos={0 0}, upload=138545]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Dec 12 05:52:35.522242732 (158h23m35.393325777s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 12 05:52:35.522248944 (158h23m35.393331989s) [upload=138545]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 12 05:52:35.844205119 (158h23m35.715288164s) [upload=138545]: Done uploading segments...
Dec 12 05:52:35.844231129 (158h23m35.715314164s) [upload=138545]: Upload finished. err: <nil>
Duration: 6.392351226s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................]
duration:
5.718923843s
trace:
https://jaeger.dev.storj.io/trace/76c0b0a8c78b2475
FAIL (2025-12-12 06:33:12 UTC)
orbiter warning: operation: ListObjects
uplink: metainfo: metaclient: context deadline exceeded
Duration: 5.718956315s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................]
duration:
5.936328839s
trace:
https://jaeger.dev.storj.io/trace/76c0b0a8c78b2475
FAIL (2025-12-12 06:32:54 UTC)
orbiter warning: operation: ListObjects
uplink: metainfo: metaclient: context deadline exceeded
Duration: 5.936357283s core uplink us1