Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.257314315s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:09:19 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.257344193s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.829784175s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:08:27 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.829814672s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.9678801s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:08:01 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.967970221s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.553870456s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:07:34 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.55389863s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.019390733s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:06:59 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.019425858s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.168430834s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:06:42 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.168463647s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.943882974s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:06:18 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.943910656s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.554610014s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:05:07 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.554652356s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.495719339s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:04:31 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.495745581s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.80456859s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:03:47 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.804597424s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.761334143s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:03:30 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.761361554s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.758739572s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:02:50 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.758769931s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.005859798s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:02:06 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.005887902s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.931960926s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:01:33 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.931985592s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.900294395s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:00:52 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.900322518s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 7.533673366s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 09:00:11 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 7.533734671s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.265794591s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 08:59:40 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.265822455s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.054504799s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 08:59:03 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.054536679s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.237498915s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 08:58:12 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.237609877s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.038685631s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 08:57:24 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.038714196s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.827234879s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 08:56:57 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.827264664s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.721105403s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 08:56:14 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.721134417s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.935543242s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 08:55:40 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.93556796s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 10.422834845s trace: https://jaeger.dev.storj.io/trace/3fc4fbd559feeeb8 FAIL (2025-11-23 08:55:23 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 10.422862339s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.744589938s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:31:07 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.744620688s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.698488334s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:30:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.698521647s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.681864236s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:29:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.681891879s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.717129086s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:28:11 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.71715216s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.622642283s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:26:59 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.6226724s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.723262748s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:26:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.723297827s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.675967692s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:24:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.675996427s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.64320621s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:23:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.643239804s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.702702668s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:23:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.702730883s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.712764029s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:22:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.712800698s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.651448799s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:20:57 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.6514808s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.634417741s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:20:22 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.634439723s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.730403306s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:19:23 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.730429956s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.707502587s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:18:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.707531963s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.775645067s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:17:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.775673029s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.619376755s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:17:15 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.619407323s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.762676206s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:16:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.762707175s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.693083102s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:15:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.693112247s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.830899456s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:15:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.830922991s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.806283372s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:14:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.806311596s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.782030578s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:13:09 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.78206409s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.499816544s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:12:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.499846291s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.74194969s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:11:13 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.741982523s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.611016935s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:10:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.611053946s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.767329501s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:09:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.767354808s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.806457883s trace: https://jaeger.dev.storj.io/trace/3a6b630ef11b65dd FAIL (2025-11-23 07:09:07 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2:45.289113605 (334h38m43.840584066s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.289593559 (334h38m43.84106402s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290002558 (334h38m43.841473039s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290292069 (334h38m43.84176254s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290335582 (334h38m43.841806053s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290718160 (334h38m43.842188621s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.290725214 (334h38m43.842195675s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291047217 (334h38m43.842517668s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291097112 (334h38m43.842567573s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291374640 (334h38m43.842845101s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291481193 (334h38m43.842951654s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291694330 (334h38m43.843164781s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291852120 (334h38m43.843322581s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.291997056 (334h38m43.843467517s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292212576 (334h38m43.843683037s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.292565929 (334h38m43.84403639s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:45.293218471 (334h38m43.844688922s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Uploading piece... Nov 23 06:42:46.697488402 (334h38m45.248958863s) [upload=294625]: Next returned nil segment Nov 23 06:42:46.697554037 (334h38m45.249024518s) [upload=294625]: Waiting for error group managing segments... Nov 23 06:42:47.078563880 (334h38m45.630034341s) [noise=true , address=64.49.13.140:4514 , piece_id=RRPEPRHLQKCO6GW6YKWH52AT5V27ZZRC6AXXN4WNYBTZ4ADLYEHQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.142373564 (334h38m45.693844015s) [noise=true , address=145.79.227.52:4473 , piece_id=OMHDLY4JHII47N3VWCMI7TU3K3GRVVYLBCVNVCKW5WEDR5QADH6Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.146746771 (334h38m45.698217232s) [noise=true , address=145.79.227.46:4619 , piece_id=XXB2ZUBMF5OM7XKHS2Y2WXTQA2B35TX24OHHPCDQPGSFUDFXCBVA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.147423840 (334h38m45.698894311s) [noise=true , address=145.79.227.15:4893 , piece_id=FCJUFZ6W5KRL4V4TDNSMRCXOKWDNQFZAYW7TCTZZLHCUMCUP5UTA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.165172977 (334h38m45.716643428s) [noise=true , address=145.79.227.6:4265 , piece_id=2TTHVLKLUZGNEQXTHEECAAVQ24TAQH5SRE7OQH7RQOC2OSOB7WRQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.194913648 (334h38m45.746384109s) [noise=true , address=45.91.116.13:4164 , piece_id=WO22V3SLOI7DW7DZLOHKVBJ3NQ7XYPVTKNYFUWYI5NYJJZDHUTYQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.204840269 (334h38m45.756310721s) [noise=true , address=45.91.116.9:4274 , piece_id=IWMXQQVGNJPN5ZUJIDMMO2OPHDW6TUELOLUTLHHUURQSTTYWTEQQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.245478521 (334h38m45.796948982s) [noise=true , address=45.91.116.8:4061 , piece_id=ML3GSGMR4JMKV7MQX5YS4TWSNQJFXEYQDQOKYRKJV3XUMPO4OBMA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.249777929 (334h38m45.80124838s) [noise=true , address=77.90.31.19:4853 , piece_id=LG3WF7CDPWWVRTAOSYORQEC4LEZMDJ3MCK4K55JEQCG5SGMNNZDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.274814422 (334h38m45.826284883s) [noise=true , address=77.90.31.10:4530 , piece_id=VOAAXSSTOQDYU2F4PIYWOVD2TASLFLIRW4MNWK7LWBDT4JJLQYSA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.281648856 (334h38m45.833119317s) [noise=true , address=45.91.116.10:4101 , piece_id=GYDAAM2BNUQ4IWZVKC6H2KUCHOXVUHGCBGZLXO6RNOLNAO3XK2XA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.289895218 (334h38m45.8413657s) [noise=true , address=45.91.116.14:4209 , piece_id=FC6THM3L5NSFK24TSYV67SIY5Q7QXOKQZHONOKGML57TPZIFCWDA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.294443259 (334h38m45.84591372s) [noise=true , address=77.90.31.18:4815 , piece_id=GJINVQS42ETZDWPLZ2C27NXBYO45B4OBBJBUCQKD5TNZ4RQJ5TEA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300527985 (334h38m45.851998446s) [noise=true , address=77.90.31.16:4750 , piece_id=OAJI5CMTADUT3OQXTTPTYVWOAS4F2OZH7MTODJMDUQ6HCVCTQBMQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: <nil> Nov 23 06:42:47.300552221 (334h38m45.852022682s) [seg_pos={0 0}, upload=294625]: Segment reached optimal threshold of 14 pieces. Nov 23 06:42:47.300867952 (334h38m45.852338413s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123HQXrHXyyMYHd5j7nqjzdGWoNg6m8j1eejUcvHNRfCRaHjAWK): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.300879825 (334h38m45.852350297s) [noise=true , address=77.90.31.17:4798 , piece_id=24UM2YD25V6XNNJNRI56IF2CYXCEEBFUNL5ZS6T6UPWYDXRHXBUQ, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.300988051 (334h38m45.852458513s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CEXqfegKNoaZ516cRAYvaMBygJVdMtvyrArc85zK3eDQC6tXN): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301008430 (334h38m45.852478891s) [noise=true , address=77.90.31.13:4656 , piece_id=UPEGTOY4G4BOTVTXWZZNKXR4WX74FE7FTQ6J5BHHWHBCJ3KVNA3Q, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301075207 (334h38m45.852545678s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YZZnCo54XknuvoBoVtRudXSnoWdNdrn23Lp6Gou3g451BzUwG): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301086228 (334h38m45.852556689s) [noise=true , address=77.90.31.15:4706 , piece_id=EHHTQVTKVAAUPI5YFEEKDU4BCESDJ3S5ZRVOUI7XQXUF7TQ6AVIA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301290116 (334h38m45.852760577s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1oZ1kyuE2AMYqcQbQpnn6T5pVbxbcfXkDe1MMKUqAYFMPfTm1u): protocol: expected piece hash; context canceled; EOF Nov 23 06:42:47.301300066 (334h38m45.852770527s) [noise=true , address=45.91.116.6:4979 , piece_id=FVJTPADGSW5FI4SEC3FDXGK7IDOTE7WK23XALFANUSZSP67YHTXA, seg_pos={0 0}, upload=294625]: Long tail cancellation due to optimal threshold Nov 23 06:42:47.301373245 (334h38m45.852843706s) [seg_pos={0 0}, upload=294625]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 23 06:42:47.301465340 (334h38m45.852935801s) [upload=294625]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 23 06:42:47.301472684 (334h38m45.852943145s) [upload=294625]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 23 06:42:47.595414676 (334h38m46.146885147s) [upload=294625]: Done uploading segments... Nov 23 06:42:47.595473598 (334h38m46.146944059s) [upload=294625]: Upload finished. err: <nil> Duration: 5.806496407s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)