Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
6.785962307s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:44:47 UTC)
orbiter warning: context deadline exceeded
Duration: 6.785989058s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
6.567622779s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:44:20 UTC)
orbiter warning: context deadline exceeded
Duration: 6.567653336s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
6.785927403s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:43:52 UTC)
orbiter warning: context deadline exceeded
Duration: 6.785965135s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
7.309800764s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:43:22 UTC)
orbiter warning: context deadline exceeded
Duration: 7.309822937s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
7.291418581s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:43:04 UTC)
orbiter warning: context deadline exceeded
Duration: 7.29146013s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
6.665058655s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:42:12 UTC)
orbiter warning: context deadline exceeded
Duration: 6.66507185s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
7.6424378s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:41:36 UTC)
orbiter warning: context deadline exceeded
Duration: 7.642467365s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
6.824073862s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:40:24 UTC)
orbiter warning: context deadline exceeded
Duration: 6.824100023s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
6.622659908s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:39:53 UTC)
orbiter warning: context deadline exceeded
Duration: 6.622702691s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
6.638932234s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:38:59 UTC)
orbiter warning: context deadline exceeded
Duration: 6.638959845s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
6.820402867s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:37:48 UTC)
orbiter warning: context deadline exceeded
Duration: 6.820459015s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
6.842818575s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:36:50 UTC)
orbiter warning: context deadline exceeded
Duration: 6.842866888s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
6.872426349s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:36:10 UTC)
orbiter warning: context deadline exceeded
Duration: 6.872441047s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
6.878881409s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:35:44 UTC)
orbiter warning: context deadline exceeded
Duration: 6.878901999s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
6.666212023s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:35:01 UTC)
orbiter warning: context deadline exceeded
Duration: 6.666307495s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
6.792290299s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:33:47 UTC)
orbiter warning: context deadline exceeded
Duration: 6.792318384s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
6.914651438s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:33:22 UTC)
orbiter warning: context deadline exceeded
Duration: 6.91467369s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
6.803263024s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:32:30 UTC)
orbiter warning: context deadline exceeded
Duration: 6.803287562s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
6.583244811s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:32:02 UTC)
orbiter warning: context deadline exceeded
Duration: 6.583273646s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
6.811455465s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:30:53 UTC)
orbiter warning: context deadline exceeded
Duration: 6.8114744s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
6.784483741s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:29:54 UTC)
orbiter warning: context deadline exceeded
Duration: 6.784538065s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
6.838633072s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:29:18 UTC)
orbiter warning: context deadline exceeded
Duration: 6.838657388s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
6.846182608s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:28:58 UTC)
orbiter warning: context deadline exceeded
Duration: 6.846202267s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
6.725559944s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:28:14 UTC)
orbiter warning: context deadline exceeded
Duration: 6.725587508s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
6.571236439s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:27:36 UTC)
orbiter warning: context deadline exceeded
Duration: 6.571427975s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
6.779263734s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:26:42 UTC)
orbiter warning: context deadline exceeded
Duration: 6.779290615s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
6.673277755s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:25:50 UTC)
orbiter warning: context deadline exceeded
Duration: 6.673310488s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
6.88865083s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:24:40 UTC)
orbiter warning: context deadline exceeded
Duration: 6.888677692s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
6.607946869s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:24:19 UTC)
orbiter warning: context deadline exceeded
Duration: 6.607967088s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
6.668586512s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:23:53 UTC)
orbiter warning: context deadline exceeded
Duration: 6.668612783s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
6.818802718s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:23:01 UTC)
orbiter warning: context deadline exceeded
Duration: 6.818826122s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
6.56771902s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:22:15 UTC)
orbiter warning: context deadline exceeded
Duration: 6.567745692s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
5.714163076s trace:
https://jaeger.dev.storj.io/trace/793bc7a1d93181c2
FAIL (2026-05-14 05:21:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
in
May 14 04:37:54.692610915 (321h50m5.21343466s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 04:37:54.930969401 (321h50m5.451793147s) [seg_pos={0 0}, upload=308354]: Begin upload segment...
May 14 04:37:54.931049184 (321h50m5.451872979s) [seg_pos={0 0}, upload=308354]: Done begin upload segment.
May 14 04:37:54.931156629 (321h50m5.451980374s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931187608 (321h50m5.452011383s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931318638 (321h50m5.452142383s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931998635 (321h50m5.45282238s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932196884 (321h50m5.453020669s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932509510 (321h50m5.453333255s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932689223 (321h50m5.453512968s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933251857 (321h50m5.454075612s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933832485 (321h50m5.45465623s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934174568 (321h50m5.454998313s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934314514 (321h50m5.455138259s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934671185 (321h50m5.45549494s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934686375 (321h50m5.45551013s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935183793 (321h50m5.456007538s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935759361 (321h50m5.456583136s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.936232875 (321h50m5.45705662s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:56.659124982 (321h50m7.179948717s) [upload=308354]: Next returned nil segment
May 14 04:37:56.659141724 (321h50m7.179965459s) [upload=308354]: Waiting for error group managing segments...
May 14 04:37:56.945082708 (321h50m7.465906453s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960510078 (321h50m7.481333823s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960815721 (321h50m7.481639466s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.963554487 (321h50m7.484378232s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964875267 (321h50m7.485699012s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964957885 (321h50m7.48578163s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.973697743 (321h50m7.494521478s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.976134162 (321h50m7.496957907s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.978984722 (321h50m7.499808457s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.017257796 (321h50m7.538081541s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.024480008 (321h50m7.545303753s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.027084668 (321h50m7.547908413s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044769395 (321h50m7.56559314s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044787289 (321h50m7.565611024s) [seg_pos={0 0}, upload=308354]: Satisfied cohort requirements after 13 pieces
May 14 04:37:57.044949048 (321h50m7.565772793s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN): protocol: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection
May 14 04:37:57.044953506 (321h50m7.565777251s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045112039 (321h50m7.565935774s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection
May 14 04:37:57.045119323 (321h50m7.565943068s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045126506 (321h50m7.565950251s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection
May 14 04:37:57.045133800 (321h50m7.565957545s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045209124 (321h50m7.566032869s) [seg_pos={0 0}, upload=308354]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 14 04:37:57.045309666 (321h50m7.566133411s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 14 04:37:57.045314124 (321h50m7.566137869s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 14 04:38:01.690068805 (321h50m12.21089255s) [upload=308354]: Done uploading segments...
May 14 04:38:01.690154709 (321h50m12.210978454s) [upload=308354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.714180078s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
6.29607657s trace:
https://jaeger.dev.storj.io/trace/793bc7a1d93181c2
FAIL (2026-05-14 05:21:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
in
May 14 04:37:54.692610915 (321h50m5.21343466s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 04:37:54.930969401 (321h50m5.451793147s) [seg_pos={0 0}, upload=308354]: Begin upload segment...
May 14 04:37:54.931049184 (321h50m5.451872979s) [seg_pos={0 0}, upload=308354]: Done begin upload segment.
May 14 04:37:54.931156629 (321h50m5.451980374s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931187608 (321h50m5.452011383s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931318638 (321h50m5.452142383s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931998635 (321h50m5.45282238s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932196884 (321h50m5.453020669s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932509510 (321h50m5.453333255s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932689223 (321h50m5.453512968s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933251857 (321h50m5.454075612s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933832485 (321h50m5.45465623s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934174568 (321h50m5.454998313s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934314514 (321h50m5.455138259s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934671185 (321h50m5.45549494s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934686375 (321h50m5.45551013s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935183793 (321h50m5.456007538s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935759361 (321h50m5.456583136s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.936232875 (321h50m5.45705662s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:56.659124982 (321h50m7.179948717s) [upload=308354]: Next returned nil segment
May 14 04:37:56.659141724 (321h50m7.179965459s) [upload=308354]: Waiting for error group managing segments...
May 14 04:37:56.945082708 (321h50m7.465906453s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960510078 (321h50m7.481333823s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960815721 (321h50m7.481639466s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.963554487 (321h50m7.484378232s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964875267 (321h50m7.485699012s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964957885 (321h50m7.48578163s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.973697743 (321h50m7.494521478s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.976134162 (321h50m7.496957907s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.978984722 (321h50m7.499808457s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.017257796 (321h50m7.538081541s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.024480008 (321h50m7.545303753s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.027084668 (321h50m7.547908413s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044769395 (321h50m7.56559314s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044787289 (321h50m7.565611024s) [seg_pos={0 0}, upload=308354]: Satisfied cohort requirements after 13 pieces
May 14 04:37:57.044949048 (321h50m7.565772793s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN): protocol: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection
May 14 04:37:57.044953506 (321h50m7.565777251s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045112039 (321h50m7.565935774s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection
May 14 04:37:57.045119323 (321h50m7.565943068s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045126506 (321h50m7.565950251s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection
May 14 04:37:57.045133800 (321h50m7.565957545s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045209124 (321h50m7.566032869s) [seg_pos={0 0}, upload=308354]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 14 04:37:57.045309666 (321h50m7.566133411s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 14 04:37:57.045314124 (321h50m7.566137869s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 14 04:38:01.690068805 (321h50m12.21089255s) [upload=308354]: Done uploading segments...
May 14 04:38:01.690154709 (321h50m12.210978454s) [upload=308354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.296097531s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
5.7200074s trace:
https://jaeger.dev.storj.io/trace/793bc7a1d93181c2
FAIL (2026-05-14 05:21:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
in
May 14 04:37:54.692610915 (321h50m5.21343466s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 04:37:54.930969401 (321h50m5.451793147s) [seg_pos={0 0}, upload=308354]: Begin upload segment...
May 14 04:37:54.931049184 (321h50m5.451872979s) [seg_pos={0 0}, upload=308354]: Done begin upload segment.
May 14 04:37:54.931156629 (321h50m5.451980374s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931187608 (321h50m5.452011383s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931318638 (321h50m5.452142383s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931998635 (321h50m5.45282238s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932196884 (321h50m5.453020669s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932509510 (321h50m5.453333255s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932689223 (321h50m5.453512968s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933251857 (321h50m5.454075612s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933832485 (321h50m5.45465623s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934174568 (321h50m5.454998313s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934314514 (321h50m5.455138259s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934671185 (321h50m5.45549494s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934686375 (321h50m5.45551013s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935183793 (321h50m5.456007538s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935759361 (321h50m5.456583136s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.936232875 (321h50m5.45705662s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:56.659124982 (321h50m7.179948717s) [upload=308354]: Next returned nil segment
May 14 04:37:56.659141724 (321h50m7.179965459s) [upload=308354]: Waiting for error group managing segments...
May 14 04:37:56.945082708 (321h50m7.465906453s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960510078 (321h50m7.481333823s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960815721 (321h50m7.481639466s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.963554487 (321h50m7.484378232s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964875267 (321h50m7.485699012s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964957885 (321h50m7.48578163s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.973697743 (321h50m7.494521478s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.976134162 (321h50m7.496957907s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.978984722 (321h50m7.499808457s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.017257796 (321h50m7.538081541s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.024480008 (321h50m7.545303753s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.027084668 (321h50m7.547908413s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044769395 (321h50m7.56559314s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044787289 (321h50m7.565611024s) [seg_pos={0 0}, upload=308354]: Satisfied cohort requirements after 13 pieces
May 14 04:37:57.044949048 (321h50m7.565772793s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN): protocol: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection
May 14 04:37:57.044953506 (321h50m7.565777251s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045112039 (321h50m7.565935774s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection
May 14 04:37:57.045119323 (321h50m7.565943068s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045126506 (321h50m7.565950251s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection
May 14 04:37:57.045133800 (321h50m7.565957545s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045209124 (321h50m7.566032869s) [seg_pos={0 0}, upload=308354]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 14 04:37:57.045309666 (321h50m7.566133411s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 14 04:37:57.045314124 (321h50m7.566137869s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 14 04:38:01.690068805 (321h50m12.21089255s) [upload=308354]: Done uploading segments...
May 14 04:38:01.690154709 (321h50m12.210978454s) [upload=308354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.720034151s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
6.645374556s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:21:01 UTC)
orbiter warning: context deadline exceeded
Duration: 6.645405324s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [X..X........................................................] duration:
6.238332796s trace:
https://jaeger.dev.storj.io/trace/793bc7a1d93181c2
FAIL (2026-05-14 05:20:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
in
May 14 04:37:54.692610915 (321h50m5.21343466s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 04:37:54.930969401 (321h50m5.451793147s) [seg_pos={0 0}, upload=308354]: Begin upload segment...
May 14 04:37:54.931049184 (321h50m5.451872979s) [seg_pos={0 0}, upload=308354]: Done begin upload segment.
May 14 04:37:54.931156629 (321h50m5.451980374s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931187608 (321h50m5.452011383s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931318638 (321h50m5.452142383s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931998635 (321h50m5.45282238s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932196884 (321h50m5.453020669s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932509510 (321h50m5.453333255s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932689223 (321h50m5.453512968s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933251857 (321h50m5.454075612s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933832485 (321h50m5.45465623s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934174568 (321h50m5.454998313s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934314514 (321h50m5.455138259s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934671185 (321h50m5.45549494s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934686375 (321h50m5.45551013s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935183793 (321h50m5.456007538s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935759361 (321h50m5.456583136s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.936232875 (321h50m5.45705662s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:56.659124982 (321h50m7.179948717s) [upload=308354]: Next returned nil segment
May 14 04:37:56.659141724 (321h50m7.179965459s) [upload=308354]: Waiting for error group managing segments...
May 14 04:37:56.945082708 (321h50m7.465906453s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960510078 (321h50m7.481333823s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960815721 (321h50m7.481639466s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.963554487 (321h50m7.484378232s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964875267 (321h50m7.485699012s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964957885 (321h50m7.48578163s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.973697743 (321h50m7.494521478s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.976134162 (321h50m7.496957907s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.978984722 (321h50m7.499808457s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.017257796 (321h50m7.538081541s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.024480008 (321h50m7.545303753s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.027084668 (321h50m7.547908413s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044769395 (321h50m7.56559314s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044787289 (321h50m7.565611024s) [seg_pos={0 0}, upload=308354]: Satisfied cohort requirements after 13 pieces
May 14 04:37:57.044949048 (321h50m7.565772793s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN): protocol: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection
May 14 04:37:57.044953506 (321h50m7.565777251s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045112039 (321h50m7.565935774s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection
May 14 04:37:57.045119323 (321h50m7.565943068s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045126506 (321h50m7.565950251s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection
May 14 04:37:57.045133800 (321h50m7.565957545s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045209124 (321h50m7.566032869s) [seg_pos={0 0}, upload=308354]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 14 04:37:57.045309666 (321h50m7.566133411s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 14 04:37:57.045314124 (321h50m7.566137869s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 14 04:38:01.690068805 (321h50m12.21089255s) [upload=308354]: Done uploading segments...
May 14 04:38:01.690154709 (321h50m12.210978454s) [upload=308354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.238377291s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.X..X.......................................................] duration:
5.84476207s trace:
https://jaeger.dev.storj.io/trace/793bc7a1d93181c2
FAIL (2026-05-14 05:20:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
in
May 14 04:37:54.692610915 (321h50m5.21343466s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 04:37:54.930969401 (321h50m5.451793147s) [seg_pos={0 0}, upload=308354]: Begin upload segment...
May 14 04:37:54.931049184 (321h50m5.451872979s) [seg_pos={0 0}, upload=308354]: Done begin upload segment.
May 14 04:37:54.931156629 (321h50m5.451980374s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931187608 (321h50m5.452011383s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931318638 (321h50m5.452142383s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931998635 (321h50m5.45282238s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932196884 (321h50m5.453020669s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932509510 (321h50m5.453333255s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932689223 (321h50m5.453512968s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933251857 (321h50m5.454075612s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933832485 (321h50m5.45465623s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934174568 (321h50m5.454998313s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934314514 (321h50m5.455138259s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934671185 (321h50m5.45549494s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934686375 (321h50m5.45551013s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935183793 (321h50m5.456007538s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935759361 (321h50m5.456583136s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.936232875 (321h50m5.45705662s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:56.659124982 (321h50m7.179948717s) [upload=308354]: Next returned nil segment
May 14 04:37:56.659141724 (321h50m7.179965459s) [upload=308354]: Waiting for error group managing segments...
May 14 04:37:56.945082708 (321h50m7.465906453s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960510078 (321h50m7.481333823s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960815721 (321h50m7.481639466s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.963554487 (321h50m7.484378232s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964875267 (321h50m7.485699012s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964957885 (321h50m7.48578163s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.973697743 (321h50m7.494521478s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.976134162 (321h50m7.496957907s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.978984722 (321h50m7.499808457s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.017257796 (321h50m7.538081541s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.024480008 (321h50m7.545303753s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.027084668 (321h50m7.547908413s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044769395 (321h50m7.56559314s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044787289 (321h50m7.565611024s) [seg_pos={0 0}, upload=308354]: Satisfied cohort requirements after 13 pieces
May 14 04:37:57.044949048 (321h50m7.565772793s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN): protocol: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection
May 14 04:37:57.044953506 (321h50m7.565777251s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045112039 (321h50m7.565935774s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection
May 14 04:37:57.045119323 (321h50m7.565943068s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045126506 (321h50m7.565950251s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection
May 14 04:37:57.045133800 (321h50m7.565957545s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045209124 (321h50m7.566032869s) [seg_pos={0 0}, upload=308354]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 14 04:37:57.045309666 (321h50m7.566133411s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 14 04:37:57.045314124 (321h50m7.566137869s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 14 04:38:01.690068805 (321h50m12.21089255s) [upload=308354]: Done uploading segments...
May 14 04:38:01.690154709 (321h50m12.210978454s) [upload=308354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.8447892s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
6.691131303s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:20:10 UTC)
orbiter warning: context deadline exceeded
Duration: 6.691154228s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
6.755482638s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:19:42 UTC)
orbiter warning: context deadline exceeded
Duration: 6.755511713s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X..X......................................................] duration:
5.57845482s trace:
https://jaeger.dev.storj.io/trace/793bc7a1d93181c2
FAIL (2026-05-14 05:19:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
in
May 14 04:37:54.692610915 (321h50m5.21343466s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 04:37:54.930969401 (321h50m5.451793147s) [seg_pos={0 0}, upload=308354]: Begin upload segment...
May 14 04:37:54.931049184 (321h50m5.451872979s) [seg_pos={0 0}, upload=308354]: Done begin upload segment.
May 14 04:37:54.931156629 (321h50m5.451980374s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931187608 (321h50m5.452011383s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931318638 (321h50m5.452142383s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931998635 (321h50m5.45282238s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932196884 (321h50m5.453020669s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932509510 (321h50m5.453333255s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932689223 (321h50m5.453512968s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933251857 (321h50m5.454075612s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933832485 (321h50m5.45465623s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934174568 (321h50m5.454998313s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934314514 (321h50m5.455138259s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934671185 (321h50m5.45549494s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934686375 (321h50m5.45551013s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935183793 (321h50m5.456007538s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935759361 (321h50m5.456583136s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.936232875 (321h50m5.45705662s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:56.659124982 (321h50m7.179948717s) [upload=308354]: Next returned nil segment
May 14 04:37:56.659141724 (321h50m7.179965459s) [upload=308354]: Waiting for error group managing segments...
May 14 04:37:56.945082708 (321h50m7.465906453s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960510078 (321h50m7.481333823s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960815721 (321h50m7.481639466s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.963554487 (321h50m7.484378232s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964875267 (321h50m7.485699012s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964957885 (321h50m7.48578163s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.973697743 (321h50m7.494521478s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.976134162 (321h50m7.496957907s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.978984722 (321h50m7.499808457s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.017257796 (321h50m7.538081541s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.024480008 (321h50m7.545303753s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.027084668 (321h50m7.547908413s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044769395 (321h50m7.56559314s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044787289 (321h50m7.565611024s) [seg_pos={0 0}, upload=308354]: Satisfied cohort requirements after 13 pieces
May 14 04:37:57.044949048 (321h50m7.565772793s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN): protocol: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection
May 14 04:37:57.044953506 (321h50m7.565777251s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045112039 (321h50m7.565935774s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection
May 14 04:37:57.045119323 (321h50m7.565943068s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045126506 (321h50m7.565950251s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection
May 14 04:37:57.045133800 (321h50m7.565957545s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045209124 (321h50m7.566032869s) [seg_pos={0 0}, upload=308354]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 14 04:37:57.045309666 (321h50m7.566133411s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 14 04:37:57.045314124 (321h50m7.566137869s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 14 04:38:01.690068805 (321h50m12.21089255s) [upload=308354]: Done uploading segments...
May 14 04:38:01.690154709 (321h50m12.210978454s) [upload=308354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.578501631s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...X..X.....................................................] duration:
6.301679844s trace:
https://jaeger.dev.storj.io/trace/793bc7a1d93181c2
FAIL (2026-05-14 05:19:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
in
May 14 04:37:54.692610915 (321h50m5.21343466s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 04:37:54.930969401 (321h50m5.451793147s) [seg_pos={0 0}, upload=308354]: Begin upload segment...
May 14 04:37:54.931049184 (321h50m5.451872979s) [seg_pos={0 0}, upload=308354]: Done begin upload segment.
May 14 04:37:54.931156629 (321h50m5.451980374s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931187608 (321h50m5.452011383s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931318638 (321h50m5.452142383s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931998635 (321h50m5.45282238s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932196884 (321h50m5.453020669s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932509510 (321h50m5.453333255s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932689223 (321h50m5.453512968s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933251857 (321h50m5.454075612s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933832485 (321h50m5.45465623s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934174568 (321h50m5.454998313s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934314514 (321h50m5.455138259s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934671185 (321h50m5.45549494s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934686375 (321h50m5.45551013s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935183793 (321h50m5.456007538s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935759361 (321h50m5.456583136s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.936232875 (321h50m5.45705662s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:56.659124982 (321h50m7.179948717s) [upload=308354]: Next returned nil segment
May 14 04:37:56.659141724 (321h50m7.179965459s) [upload=308354]: Waiting for error group managing segments...
May 14 04:37:56.945082708 (321h50m7.465906453s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960510078 (321h50m7.481333823s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960815721 (321h50m7.481639466s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.963554487 (321h50m7.484378232s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964875267 (321h50m7.485699012s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964957885 (321h50m7.48578163s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.973697743 (321h50m7.494521478s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.976134162 (321h50m7.496957907s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.978984722 (321h50m7.499808457s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.017257796 (321h50m7.538081541s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.024480008 (321h50m7.545303753s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.027084668 (321h50m7.547908413s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044769395 (321h50m7.56559314s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044787289 (321h50m7.565611024s) [seg_pos={0 0}, upload=308354]: Satisfied cohort requirements after 13 pieces
May 14 04:37:57.044949048 (321h50m7.565772793s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN): protocol: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection
May 14 04:37:57.044953506 (321h50m7.565777251s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045112039 (321h50m7.565935774s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection
May 14 04:37:57.045119323 (321h50m7.565943068s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045126506 (321h50m7.565950251s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection
May 14 04:37:57.045133800 (321h50m7.565957545s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045209124 (321h50m7.566032869s) [seg_pos={0 0}, upload=308354]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 14 04:37:57.045309666 (321h50m7.566133411s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 14 04:37:57.045314124 (321h50m7.566137869s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 14 04:38:01.690068805 (321h50m12.21089255s) [upload=308354]: Done uploading segments...
May 14 04:38:01.690154709 (321h50m12.210978454s) [upload=308354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.301699911s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
5.821041219s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:19:06 UTC)
orbiter warning: context deadline exceeded
Duration: 5.821066266s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
638.748876ms remote:
"tcp:35.209.24.65:443" target:
https://version.storj.io/ trace:
https://jaeger.dev.storj.io/trace/61406304dd190e9e
FAIL (2026-05-14 05:18:45 UTC)
orbiter warning: Get "https://version.storj.io/": read tcp 188.166.225.170:46460->35.209.24.65:443: read: connection reset by peer
Duration: 638.777492ms
core
website
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
5.816185017s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:18:26 UTC)
orbiter warning: context deadline exceeded
Duration: 5.816209694s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
635.960753ms remote:
"tcp:35.209.24.65:443" target:
https://version.storj.io/ trace:
https://jaeger.dev.storj.io/trace/61406304dd190e9e
FAIL (2026-05-14 05:18:08 UTC)
orbiter warning: Get "https://version.storj.io/": read tcp 188.166.225.170:46460->35.209.24.65:443: read: connection reset by peer
Duration: 635.986013ms
core
website
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....X..X....................................................] duration:
5.628391574s trace:
https://jaeger.dev.storj.io/trace/793bc7a1d93181c2
FAIL (2026-05-14 05:18:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
in
May 14 04:37:54.692610915 (321h50m5.21343466s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 04:37:54.930969401 (321h50m5.451793147s) [seg_pos={0 0}, upload=308354]: Begin upload segment...
May 14 04:37:54.931049184 (321h50m5.451872979s) [seg_pos={0 0}, upload=308354]: Done begin upload segment.
May 14 04:37:54.931156629 (321h50m5.451980374s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931187608 (321h50m5.452011383s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931318638 (321h50m5.452142383s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931998635 (321h50m5.45282238s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932196884 (321h50m5.453020669s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932509510 (321h50m5.453333255s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932689223 (321h50m5.453512968s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933251857 (321h50m5.454075612s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933832485 (321h50m5.45465623s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934174568 (321h50m5.454998313s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934314514 (321h50m5.455138259s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934671185 (321h50m5.45549494s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934686375 (321h50m5.45551013s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935183793 (321h50m5.456007538s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935759361 (321h50m5.456583136s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.936232875 (321h50m5.45705662s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:56.659124982 (321h50m7.179948717s) [upload=308354]: Next returned nil segment
May 14 04:37:56.659141724 (321h50m7.179965459s) [upload=308354]: Waiting for error group managing segments...
May 14 04:37:56.945082708 (321h50m7.465906453s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960510078 (321h50m7.481333823s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960815721 (321h50m7.481639466s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.963554487 (321h50m7.484378232s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964875267 (321h50m7.485699012s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964957885 (321h50m7.48578163s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.973697743 (321h50m7.494521478s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.976134162 (321h50m7.496957907s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.978984722 (321h50m7.499808457s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.017257796 (321h50m7.538081541s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.024480008 (321h50m7.545303753s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.027084668 (321h50m7.547908413s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044769395 (321h50m7.56559314s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044787289 (321h50m7.565611024s) [seg_pos={0 0}, upload=308354]: Satisfied cohort requirements after 13 pieces
May 14 04:37:57.044949048 (321h50m7.565772793s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN): protocol: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection
May 14 04:37:57.044953506 (321h50m7.565777251s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045112039 (321h50m7.565935774s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection
May 14 04:37:57.045119323 (321h50m7.565943068s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045126506 (321h50m7.565950251s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection
May 14 04:37:57.045133800 (321h50m7.565957545s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045209124 (321h50m7.566032869s) [seg_pos={0 0}, upload=308354]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 14 04:37:57.045309666 (321h50m7.566133411s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 14 04:37:57.045314124 (321h50m7.566137869s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 14 04:38:01.690068805 (321h50m12.21089255s) [upload=308354]: Done uploading segments...
May 14 04:38:01.690154709 (321h50m12.210978454s) [upload=308354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.62842013s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
637.186294ms remote:
"tcp:35.209.24.65:443" target:
https://version.storj.io/ trace:
https://jaeger.dev.storj.io/trace/61406304dd190e9e
FAIL (2026-05-14 05:17:26 UTC)
orbiter warning: Get "https://version.storj.io/": read tcp 188.166.225.170:46460->35.209.24.65:443: read: connection reset by peer
Duration: 637.21048ms
core
website
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
5.878247715s trace:
https://jaeger.dev.storj.io/trace/7e52f6d56fea8ea9
FAIL (2026-05-14 05:17:18 UTC)
orbiter warning: context deadline exceeded
Duration: 5.878301307s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....X..X...................................................] duration:
5.713699531s trace:
https://jaeger.dev.storj.io/trace/793bc7a1d93181c2
FAIL (2026-05-14 05:16:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
in
May 14 04:37:54.692610915 (321h50m5.21343466s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 04:37:54.930969401 (321h50m5.451793147s) [seg_pos={0 0}, upload=308354]: Begin upload segment...
May 14 04:37:54.931049184 (321h50m5.451872979s) [seg_pos={0 0}, upload=308354]: Done begin upload segment.
May 14 04:37:54.931156629 (321h50m5.451980374s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931187608 (321h50m5.452011383s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931318638 (321h50m5.452142383s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.931998635 (321h50m5.45282238s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932196884 (321h50m5.453020669s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932509510 (321h50m5.453333255s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.932689223 (321h50m5.453512968s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933251857 (321h50m5.454075612s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.933832485 (321h50m5.45465623s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934174568 (321h50m5.454998313s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934314514 (321h50m5.455138259s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934671185 (321h50m5.45549494s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.934686375 (321h50m5.45551013s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935183793 (321h50m5.456007538s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.935759361 (321h50m5.456583136s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:54.936232875 (321h50m5.45705662s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Uploading piece...
May 14 04:37:56.659124982 (321h50m7.179948717s) [upload=308354]: Next returned nil segment
May 14 04:37:56.659141724 (321h50m7.179965459s) [upload=308354]: Waiting for error group managing segments...
May 14 04:37:56.945082708 (321h50m7.465906453s) [noise=true , address=145.79.227.25:4618 , piece_id=E2QGMZATAZ6GS7DPQMPAJC7TUADR7CDCCJIYY234WGW52USXV5SA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960510078 (321h50m7.481333823s) [noise=true , address=145.79.227.37:4566 , piece_id=RSMGGNYFY5VIWNS6FLHFCVKVOPTQP5QUL7E5LRG6AKWDVF2I3XAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.960815721 (321h50m7.481639466s) [noise=true , address=145.79.227.45:4230 , piece_id=L3UN5CZ4ZGTUOAPG6CQY4BNYHI2EEE65LRGARQXYIOOIABW4D2FQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.963554487 (321h50m7.484378232s) [noise=true , address=145.79.227.4:4190 , piece_id=Z2KPHHRAEFXQ54GO735E7RUJ64MERQY2NTHTWMXL6TYC4YG2W2IQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964875267 (321h50m7.485699012s) [noise=true , address=145.79.227.10:4405 , piece_id=3EBKQQDFUNITALD5I4QMTOL66EEMKNON5PQE3IRZZFTSJF5NWZ5Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.964957885 (321h50m7.48578163s) [noise=true , address=145.79.227.8:4332 , piece_id=QDDEHIDSKQQX7N4LVSCWGE4TUVCTYTH5T5KMMI547PJCZ7WMLBFA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.973697743 (321h50m7.494521478s) [noise=true , address=145.79.227.23:4546 , piece_id=3RJL2JPERMTZ5AGIEHJ2M5FXBDTSC6I4BNWLAWJSZQ5LUW7IY5HA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.976134162 (321h50m7.496957907s) [noise=true , address=145.79.227.21:4477 , piece_id=5MNJ3FKZH45LM6NJEWOMUVCAYXD44G4DMPBN53M5SY7KDO6FKZYQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:56.978984722 (321h50m7.499808457s) [noise=true , address=145.79.227.39:4013 , piece_id=O6XTGBJWAOB2537CRM3EQQSQUOAIJDP6ARCXEVIGHD7GI6ROOSAA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.017257796 (321h50m7.538081541s) [noise=true , address=145.79.227.48:4306 , piece_id=PXQCVJDKUWZTAYPON6DXBOEMV2GLQOD76HXZWZHWZEABFIMFAM7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.024480008 (321h50m7.545303753s) [noise=true , address=145.79.227.12:4771 , piece_id=6QYCXP4TRPQ3Q2UXFHOL5HA22ZAJWHLZYN33ILMHO4GHWTVHAQLA, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.027084668 (321h50m7.547908413s) [noise=true , address=145.79.227.51:4412 , piece_id=3AWOXCPOPTF6C3W4IVZTF6SAP44UUBESVOJHP457FJ66EJII7QZQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044769395 (321h50m7.56559314s) [noise=true , address=145.79.227.40:4054 , piece_id=K6JH7YWWHNRUR67J4JABEKROHEM56PM2VEXQWGKVPXBIHA3VBG4A, seg_pos={0 0}, upload=308354]: Done uploading piece. err: <nil>
May 14 04:37:57.044787289 (321h50m7.565611024s) [seg_pos={0 0}, upload=308354]: Satisfied cohort requirements after 13 pieces
May 14 04:37:57.044949048 (321h50m7.565772793s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tdXgFPYY7uaVB2rqi5pTctyZpmXQcCpyMTkHy9q1r3H2T1BZN): protocol: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57844->45.91.116.8:4080: use of closed network connection
May 14 04:37:57.044953506 (321h50m7.565777251s) [noise=true , address=45.91.116.8:4080 , piece_id=5THBVBHXGFAC4PR6EH2RKERCO4U7JA2IKAMGWGFICDQLE372XSVQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045112039 (321h50m7.565935774s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW): protocol: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37446->45.91.116.5:4297: use of closed network connection
May 14 04:37:57.045119323 (321h50m7.565943068s) [noise=true , address=45.91.116.5:4297 , piece_id=SART3ZNQLIYXFX2Y5IZVAQMEVMHK4MXBXPXZT2X7D7TBWMA4OXOQ, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045126506 (321h50m7.565950251s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41754->45.91.116.10:4082: use of closed network connection
May 14 04:37:57.045133800 (321h50m7.565957545s) [noise=true , address=45.91.116.10:4082 , piece_id=HSVVFNYPI52SM3QRPBKTKRTQY6W3PGK2IMA46FUSQETPXWP4XH7Q, seg_pos={0 0}, upload=308354]: Long tail cancellation due to optimal threshold
May 14 04:37:57.045209124 (321h50m7.566032869s) [seg_pos={0 0}, upload=308354]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
May 14 04:37:57.045309666 (321h50m7.566133411s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 14 04:37:57.045314124 (321h50m7.566137869s) [upload=308354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 14 04:38:01.690068805 (321h50m12.21089255s) [upload=308354]: Done uploading segments...
May 14 04:38:01.690154709 (321h50m12.210978454s) [upload=308354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.713727784s
core
uplink
us-select-1
|