Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................X...................] duration: 3.373531856s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:35:13 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.373566273s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [X........................................X..................] duration: 3.398291779s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:34:21 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.398324051s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.X........................................X.................] duration: 3.469847298s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:33:25 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.469873558s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..X........................................X................] duration: 3.385766234s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:32:39 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.385799689s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...X........................................X...............] duration: 3.441650403s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:32:21 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.441680089s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [....X........................................X..............] duration: 3.400147144s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:31:13 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.400168906s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....X........................................X.............] duration: 3.409638098s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:30:36 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.409664698s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......X........................................X............] duration: 3.467783681s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:30:17 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.467853373s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......X........................................X...........] duration: 3.404340289s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:29:22 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.404361469s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........X........................................X..........] duration: 3.4335794s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:28:38 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.433619886s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........X........................................X.........] duration: 3.483112048s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:27:53 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.483139931s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........X........................................X........] duration: 3.422641759s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:27:26 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.422665756s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........X........................................X.......] duration: 3.425303887s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:26:55 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.425381164s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [............X........................................X......] duration: 3.408128398s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:25:42 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.408157903s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.............X........................................X.....] duration: 3.408008273s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:25:27 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.40802753s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..............X........................................X....] duration: 3.349388815s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:24:20 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.349416197s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...............X........................................X...] duration: 3.417269513s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:23:28 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.417298188s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [................X........................................X..] duration: 3.355569728s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:22:46 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.35559669s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.................X........................................X.] duration: 3.498154261s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:22:31 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.498180151s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..................X........................................X] duration: 5.001881528s trace: https://jaeger.dev.storj.io/trace/91ceba22e3e839f FAIL (2025-11-04 10:21:59 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001896457s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................X........................................] duration: 3.338779769s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:21:12 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.338810567s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................X.......................................] duration: 3.473119276s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:20:14 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.473148223s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................X......................................] duration: 3.363677421s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:19:12 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.363707579s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................X.....................................] duration: 3.422477778s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:18:53 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.422506232s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................X....................................] duration: 3.422993236s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:17:57 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.423059592s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................X...................................] duration: 3.388919553s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:17:40 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.38894907s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [X........................X..................................] duration: 3.405342209s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:17:02 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.405371555s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.X........................X.................................] duration: 3.373884134s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:16:41 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.373912339s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..X........................X................................] duration: 3.389281763s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:16:06 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.389315537s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...X........................X...............................] duration: 3.435887461s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:15:39 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.435912s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [....X........................X..............................] duration: 3.42035788s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:14:43 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.420387024s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....X........................X.............................] duration: 3.440606108s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:14:13 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.44064369s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......X........................X............................] duration: 3.411518575s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:13:40 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.411540677s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......X........................X...........................] duration: 3.394817107s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:12:35 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.394848465s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........X........................X..........................] duration: 3.422016678s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:11:29 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.422043319s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........X........................X.........................] duration: 3.428773333s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:10:35 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.428804974s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 4.842822916s trace: https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5 FAIL (2025-11-04 10:09:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments... Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces. Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments... Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.84285623s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........X........................X........................] duration: 3.391119547s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:09:25 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.391150155s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 4.895698861s trace: https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5 FAIL (2025-11-04 10:09:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments... Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces. Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments... Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.895831754s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 4.767637405s trace: https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5 FAIL (2025-11-04 10:08:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments... Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces. Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments... Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.767666519s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........X........................X.......................] duration: 3.380681737s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:08:25 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.380710772s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 4.886488454s trace: https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5 FAIL (2025-11-04 10:08:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments... Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces. Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments... Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.88652356s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [............X........................X......................] duration: 3.386888927s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:07:46 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.386921178s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 4.893023634s trace: https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5 FAIL (2025-11-04 10:07:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments... Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces. Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments... Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.893076465s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.............X........................X.....................] duration: 3.375410212s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:06:59 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.375437424s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.464772711s trace: https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5 FAIL (2025-11-04 10:06:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments... Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces. Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments... Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.464802477s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..............X........................X....................] duration: 3.390338289s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:06:08 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.390376461s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.460817513s trace: https://jaeger.dev.storj.io/trace/fe1b57ca38f9d5 FAIL (2025-11-04 10:05:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.833784389 (117h44m7.529633142s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.834635689 (117h44m7.530484443s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.835564679 (117h44m7.531413442s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.837103248 (117h44m7.532952001s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.838931159 (117h44m7.534779902s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.840976174 (117h44m7.536824927s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.843209847 (117h44m7.53905863s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.845480721 (117h44m7.541329474s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.847906691 (117h44m7.543755444s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:35.850674971 (117h44m7.546523724s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Uploading piece... Nov 4 09:24:36.664111590 (117h44m8.359960343s) [upload=103704]: Next returned nil segment Nov 4 09:24:36.664131899 (117h44m8.359980642s) [upload=103704]: Waiting for error group managing segments... Nov 4 09:24:37.045621097 (117h44m8.74146985s) [noise=true , address=81.171.14.70:4040 , piece_id=GEPJSR6OOFLMWVK6L6P5M6EUVQKWE22KPHNVIDHMADZKFJ5FUM4Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.056210781 (117h44m8.752059534s) [noise=true , address=81.171.14.70:4033 , piece_id=ETQB3UOTGZ43POUYWXGKG3E4ME7R6RSLXOEIXS3JNTGCNS6P7TQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.060042558 (117h44m8.755891301s) [noise=true , address=81.171.14.70:4028 , piece_id=YMF72OFENI7X7H3YMB2JQ6OIRLRTTEMEXEZAQDOTYQZL5Y4S75NA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.064340322 (117h44m8.760189075s) [noise=true , address=81.171.14.70:4031 , piece_id=PNZH3UOW5IGQSEY72TRKSREPCDQQEC6FORXAVHMWXFOCFY6A4RQQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.102924393 (117h44m8.798773146s) [noise=true , address=89.149.201.147:4019 , piece_id=NXJWZY4FFD6CRVDU3PYON75MTOD3BWJKNZOA4Y7OOJZFR3DIRD3Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.105278415 (117h44m8.801127158s) [noise=true , address=81.171.14.70:4045 , piece_id=Z2RWY6FAMDKG7JNN57NO3RQSTNDQDB7MCGFQJ425KOJIF7QREDEA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.115872909 (117h44m8.811721653s) [noise=true , address=89.149.201.147:4015 , piece_id=YJEYCADFGHOOF5U4RZ4OJI7CA5LXAB3GODZTT6N4DVO57YCKMZSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.126338458 (117h44m8.822187201s) [noise=true , address=81.171.14.70:4035 , piece_id=O7VUIYG42YBUKTLL32W7PAY6EWO5VOOUIVGWQBIDS2VANVZK4HIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.191443420 (117h44m8.887292173s) [noise=true , address=89.149.201.147:4003 , piece_id=5JOP42Z6EVY6RM33OIVFFZ6RQVWOARYAKYLFBAWX5DJLATWW624Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.194241959 (117h44m8.890090712s) [noise=true , address=89.149.201.147:4008 , piece_id=RSM2A63DSMYD6OX3FW2HOF3G7EKBM2YGB4G75QIWTUBACF3H3MFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.551317990 (117h44m9.247166743s) [noise=true , address=81.171.14.70:4026 , piece_id=AJLQVEFRMVKNFTZGY5OW644BENCT23HVOSQ7GPADORG7RXHGTSNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568367696 (117h44m9.264216449s) [noise=true , address=81.171.14.70:4030 , piece_id=KEDQVSOV6REFMOK6DJOQFQ4KW2SIIHHDNK6S52TGOBUXNH47RD5Q, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.568875453 (117h44m9.264724196s) [noise=true , address=81.171.14.70:4034 , piece_id=MQ4ICAT4UQQYMW7NM5OITWDWSG25ENMTUF3GJBFYD22FOZXWQSFA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.589097079 (117h44m9.284945832s) [noise=true , address=81.171.14.70:4047 , piece_id=XV4RTQOSITSYJSOBS6Y6VHAAPGJMBOFN2S7HYJ4PIGJAWJVLRL6A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.601272223 (117h44m9.297120976s) [noise=true , address=89.149.201.147:4023 , piece_id=Q7UEKYWHTRWNIPUZPQNRIWZA5XNM5XNPCYJ44PN6NY6IUN3F32IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.609350455 (117h44m9.305199198s) [noise=true , address=89.149.201.147:4020 , piece_id=WDXS33TX2MWWF5EVV6QGD6QEWUG26TB2USF5YHHFMU5LBBCTR3IA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.629258303 (117h44m9.325107056s) [noise=true , address=185.167.204.205:4026 , piece_id=6T6FV6V432RZSVC3LAJSBWBR5K3ZEFHC5P7PRU4QOIRRVSZX7MIA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.654483166 (117h44m9.350331909s) [noise=true , address=185.167.204.206:4038 , piece_id=FPRBRF7CXQXFQ3K4BRU3S5TJHW7QUQZ3AVWB2EVGG6T743DT6U3A, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.660526313 (117h44m9.356375057s) [noise=true , address=89.149.201.147:4017 , piece_id=DGOKI3WO2NFQSRWZO46ZINJJ4DQHYQ6A2KFDMJRFP2ZWAPT2TUDA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.663703333 (117h44m9.359552076s) [noise=true , address=185.167.204.206:4047 , piece_id=3EYIA3ZWROJHPB3AHCOBKIJOJ6T53MKHC4ENWXE4WMLKQKIRRRUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.695782357 (117h44m9.39163111s) [noise=true , address=185.167.204.206:4048 , piece_id=2CDL4Y6NQP4SNSUTDUND7SYMDYVZZGH6RD4ZBYHS3YKZFIYW5RUQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.731635860 (117h44m9.427484613s) [noise=true , address=185.167.204.205:4027 , piece_id=WN4VI6SD7ZKUUPJHY3HPR7NWT654MI2BJJFNSX5DPWDVA75FA5SA, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.746959419 (117h44m9.442808172s) [noise=true , address=185.167.204.205:4003 , piece_id=4PIRLQOHQLDY5XUZX4IBJWH77VFN4K6J34DEE2KQTPWGD4HMAHLQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.752846228 (117h44m9.448694982s) [noise=true , address=185.167.204.206:4041 , piece_id=VUECGW4KJIHVCPYMUZNFDV4MLEZXYTPCVS5WDFJT5ESGG36CDQYQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.758217416 (117h44m9.454066169s) [noise=true , address=185.167.204.205:4021 , piece_id=C3VLJX3FTH5FCGL5Y544RB3NIUJQABUJB6PGPZWF5FZHUNYBEWSQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805080161 (117h44m9.500928914s) [noise=true , address=185.167.204.206:4058 , piece_id=ZQWHADQTIBRCBEKWJJWNSVP25BTN4K24A5QOZQFICSDL6PD2JIFQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: <nil> Nov 4 09:24:37.805112914 (117h44m9.500961667s) [seg_pos={0 0}, upload=103704]: Segment reached optimal threshold of 26 pieces. Nov 4 09:24:37.805362599 (117h44m9.501211352s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPbi9CNAvezq91fdgjboVcgrdwwN48dFfLXzzgXtJ9B5eEoYq): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805371898 (117h44m9.501220651s) [noise=true , address=185.167.204.206:4045 , piece_id=OVJKACGPZFX6X5MCQTE4HCIJ2YS42CWXMJXJGG6BGQANYEBS6YNQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805623827 (117h44m9.50147257s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1fT9qH1igPUNfrzpwa2VkBX7heh475sNqoN4Nsf9FYSwi1VmHj): protocol: expected piece hash; context canceled; EOF Nov 4 09:24:37.805676367 (117h44m9.50152512s) [noise=true , address=185.167.204.205:4022 , piece_id=HBXNICVQGEJD7IM6SKUF2BMRZX4NBX2PPE4P6GNHB3WP3RS2DHXQ, seg_pos={0 0}, upload=103704]: Long tail cancellation due to optimal threshold Nov 4 09:24:37.805719439 (117h44m9.501568182s) [seg_pos={0 0}, upload=103704]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Nov 4 09:24:37.805868984 (117h44m9.501717737s) [upload=103704]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 4 09:24:37.805875586 (117h44m9.501724329s) [upload=103704]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 4 09:26:21.434315496 (117h45m53.130164239s) [upload=103704]: Done uploading segments... Nov 4 09:26:21.434377824 (117h45m53.130226567s) [upload=103704]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.460846779s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...............X........................X...................] duration: 3.37439488s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:05:21 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.374428584s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [................X........................X..................] duration: 3.431680846s trace: https://jaeger.dev.storj.io/trace/4978f2fdea92fad2 FAIL (2025-11-04 10:05:01 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.431700783s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)