Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................X...............................] duration: 3.841781845s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:11:06 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.841810501s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................X..............................] duration: 3.386780861s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:10:07 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.386802182s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................X.............................] duration: 4.360454229s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:09:51 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.360475519s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................X............................] duration: 3.485162186s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:09:20 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.485182354s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................X...........................] duration: 3.485237703s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:08:17 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.485262401s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................X..........................] duration: 3.486845024s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:07:55 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.486860434s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................X.........................] duration: 3.609768149s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:07:31 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.609788558s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................X........................] duration: 3.479788762s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:07:09 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.479810847s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................X.......................] duration: 3.881413837s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:05:57 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.881433845s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................X......................] duration: 3.832576336s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:05:27 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.832589993s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................X.....................] duration: 3.719253375s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:04:41 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.71928199s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................X....................] duration: 3.773445421s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:04:15 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.773467785s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................X...................] duration: 3.859211636s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:03:36 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.859230332s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................X..................] duration: 3.796360417s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:02:38 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.796386436s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................X.................] duration: 3.809689088s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:02:03 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.809708877s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................X................] duration: 4.82712999s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 14:00:59 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.827152372s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................................X...............] duration: 3.477921775s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:59:46 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.477941955s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................................X..............] duration: 3.554865523s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:59:09 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.554879308s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................................X.............] duration: 3.63524936s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:58:32 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.635308784s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................................X............] duration: 3.42756526s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:57:47 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.427584656s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................................X...........] duration: 3.428520341s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:57:31 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.428698646s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................................X..........] duration: 3.492896503s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:56:42 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.492919679s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 3.536695835s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:55:51 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.536716063s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 3.491523619s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:55:02 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.4916379s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 3.472369106s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:53:55 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.472390887s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 3.430769282s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:53:20 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.430789982s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 3.405987644s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:52:52 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.406010648s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 3.375816072s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:52:11 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.375841251s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 4.497376443s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:51:47 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.497402882s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 3.442890063s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:50:46 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.442919119s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 3.398807307s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:49:37 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.398834039s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.000706966s trace: https://jaeger.dev.storj.io/trace/38476b60de2134f8 FAIL (2026-04-08 13:49:12 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000727235s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 11.515331157s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:28:20 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 11.515361294s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 17.211903814s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:27:13 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 17.211929534s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 10.138234831s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:25:56 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 10.138258305s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.540968636s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:24:38 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 6.540986s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 7.044668541s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:23:54 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 7.044687628s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 7.346658684s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:23:01 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 7.346685496s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 7.17638848s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:22:45 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 7.176412817s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.453586348s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:22:07 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 6.453610625s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 8.314220429s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:21:46 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 8.314266569s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 9.565658463s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:21:01 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 9.565686187s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.947912328s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:19:51 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 6.947941524s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 9.550434175s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:19:39 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 9.550459886s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 8.373847047s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:19:11 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 8.373874149s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 7.643561845s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:18:54 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 7.643583447s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 8.501779808s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:18:07 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 8.501804257s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 7.519236769s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:17:10 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 7.519254944s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 7.254014224s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:16:42 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 7.254045114s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 11.309819483s trace: https://jaeger.dev.storj.io/trace/37c1fe45e8b4c47b FAIL (2026-04-08 06:16:26 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context canceled ======LOGS====== os={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.992933342 (13h55m57.812169376s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993331870 (13h55m57.812567905s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993648392 (13h55m57.812884416s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.993728286 (13h55m57.81296431s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994007955 (13h55m57.813243989s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994266114 (13h55m57.813502138s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994580451 (13h55m57.813816475s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994655887 (13h55m57.813891911s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.994851043 (13h55m57.814087077s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995095916 (13h55m57.81433194s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995261135 (13h55m57.814497159s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995620167 (13h55m57.814856201s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.995971495 (13h55m57.815207519s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996020921 (13h55m57.815256955s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996388510 (13h55m57.815624534s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.996756680 (13h55m57.815992704s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:10.997179236 (13h55m57.81641526s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Uploading piece... Apr 8 05:42:12.335303265 (13h55m59.154539299s) [upload=13382]: Next returned nil segment Apr 8 05:42:12.335315329 (13h55m59.154551353s) [upload=13382]: Waiting for error group managing segments... Apr 8 05:42:12.710478529 (13h55m59.529714553s) [noise=true , address=167.17.72.71:4784 , piece_id=KFR4XGZ5T3AJ65TQZ5QALWJ3UI4FYDEU7EACCYQHXMOUD3QXMZXQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.735596174 (13h55m59.554832198s) [noise=true , address=64.49.13.140:4508 , piece_id=XFY5OBNCZWLIRSXO4ACYPRFWSYMJGFQWHGA74K6RXFKUII3LYSUA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.753400698 (13h55m59.572636722s) [noise=true , address=5.231.73.4:4009 , piece_id=XRVJCO4TPOJUUXV3K3OEZ6ZUC7QLUY4HDPKVUOVRHC4T2R3RHFPA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.770010905 (13h55m59.589246929s) [noise=true , address=167.17.72.70:4754 , piece_id=36KTAAASF242ZHIH7AO7BI2F2LIPQRN2JHI7QCE77BK2VFZND7HA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.787599891 (13h55m59.606835915s) [noise=true , address=167.17.72.68:4644 , piece_id=CM4DPVRP3QFXIO4EU2KYQDHIB2DX3OJQXXJ6RXG73OW5O6OIXOVQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.789010934 (13h55m59.608246958s) [noise=true , address=167.17.72.69:4733 , piece_id=UHS3AFDN26E3GMFKJ4KKVFWQO2VEB3ISCMGTQUHWQDB5PCRDIKKA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.942404141 (13h55m59.761640176s) [noise=true , address=145.79.227.17:4978 , piece_id=HS5CJ66JMJ4MMOMEKHMNDRCXOMMHLZLS4D3ODJQBBGCYDY2G7N6Q, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:12.973988539 (13h55m59.793224563s) [noise=true , address=77.90.31.13:4652 , piece_id=FO7SOKWDWOW4MLBOAVSJYXTZ5WEUQ5RFYUCYT2J6BQBT3ONIFOMA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.371952704 (13h56m0.191188728s) [noise=true , address=193.149.164.68:4058 , piece_id=ZB4JWCM6RWRNTPUG7NHJA42NL3CG5BMGN33QIBX3PX5FI4IBAVCQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.450558311 (13h56m0.269794336s) [noise=true , address=193.149.164.67:4020 , piece_id=W6LI7G74L4KP2KTUY7MYXKOGBWLHM7IAE4MU6CER74JRWVFO6M2A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.465117770 (13h56m0.284353804s) [noise=true , address=45.91.116.9:4271 , piece_id=NZUVGJ3JWLUFG4RMNYA6KJU4YMZA4TNX7MLKKNA4UQD4CQOUK2ZA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.527205759 (13h56m0.346441783s) [noise=true , address=193.149.164.66:4979 , piece_id=3MLIL65DZSMSYLX5BL4RAJFFI5MTWBIQRUSEXUYDDMCR3H4QETQA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.649983248 (13h56m0.469219273s) [noise=true , address=45.91.116.5:4312 , piece_id=WQT6OSSFCCUW7BI4MOV32TH22BFOEL2KX7T5ISSPCB6IGU6R3E6A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850896207 (13h56m0.670132231s) [noise=true , address=145.79.227.30:4775 , piece_id=INUVNOGFYI6KL5TU5EVM4OACU6FI2XXCMLVDXBUXWO36RDMZN7DA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: <nil> Apr 8 05:42:13.850914793 (13h56m0.670150817s) [seg_pos={0 0}, upload=13382]: Satisfied cohort requirements after 14 pieces Apr 8 05:42:13.851098598 (13h56m0.670334622s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kTetmDNAeCnvZFGFxdWgeChGsPonzLM5otzmKoLsFfvw4qgDr): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851104669 (13h56m0.670340693s) [noise=true , address=193.149.164.70:4117 , piece_id=DWDDPPQFYTAA7TWFUOS34FZWWNSCEFLBC4PBBBQJIZWDIO7367GA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851118225 (13h56m0.67035426s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851126701 (13h56m0.670362725s) [noise=true , address=145.79.227.33:4904 , piece_id=LTV2VMYNHOYEOBM7WP4OXBT5VLIEAR7Z2AM5VGOCEWXN5LIJJSHA, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851248948 (13h56m0.670484972s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1BNqBBBais5YbJa5pAYKWdbEWRbVm4GWEkDjkBnrTeFJvzut8c): protocol: expected piece hash; context canceled; EOF Apr 8 05:42:13.851255710 (13h56m0.670491744s) [noise=true , address=145.79.227.37:4595 , piece_id=GIBAZFJOTBCDSGITHVTOUGBJ4U64YMAKAQHPDX2V3E7BIF5QRL7A, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851383647 (13h56m0.670619671s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16hXAvToyhPMHEFW9TGrrZgjLtMMoeGA5kYQ1A7N19MY3je8qn): protocol: read tcp 188.166.225.170:41610->193.149.164.69:4090: use of closed network connection; context canceled Apr 8 05:42:13.851389629 (13h56m0.670625654s) [noise=true , address=193.149.164.69:4090 , piece_id=7PTIPOIMTWLQYMRWVWJOH3RFQ5KUYE6M3VX7RWAR5ACLEGLEUOHQ, seg_pos={0 0}, upload=13382]: Long tail cancellation due to optimal threshold Apr 8 05:42:13.851402824 (13h56m0.670638848s) [seg_pos={0 0}, upload=13382]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Apr 8 05:42:13.851464153 (13h56m0.670700177s) [upload=13382]: Flush batch item: pb.BatchRequestItem_SegmentCommit Apr 8 05:42:13.851468201 (13h56m0.670704225s) [upload=13382]: Flush batch item: pb.BatchRequestItem_ObjectCommit Apr 8 05:42:14.125209508 (13h56m0.944445542s) [upload=13382]: Done uploading segments... Apr 8 05:42:14.125228264 (13h56m0.944464288s) [upload=13382]: Upload finished. err: <nil> Duration: 11.309841775s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)