Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [.......................................................XXXXX] duration: 5.000262058s trace: https://jaeger.dev.storj.io/trace/416557f77dd945b5 FAIL (2025-11-21 11:05:39 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 5.000278961s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 802.082999ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3d967bc11165dc1c FAIL (2025-11-21 11:05:33 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": context deadline exceeded Duration: 802.113297ms core website
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 451.718871ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/5285054b3a776da7 FAIL (2025-11-21 11:05:16 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 451.746324ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 456.925086ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/e2bae25cf885024 FAIL (2025-11-21 11:05:12 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 456.972246ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 407.439668ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace: https://jaeger.dev.storj.io/trace/3e5c193367f9a974 FAIL (2025-11-21 11:05:09 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 407.471779ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 409.675074ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/41cfd75a627799ce FAIL (2025-11-21 11:05:00 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 409.694039ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 2m0.001420339s remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/41cfd75a627799ce FAIL (2025-11-21 11:04:59 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 2m0.001519568s edge linksharing bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [........................................................XXXX] duration: 5.000719387s trace: https://jaeger.dev.storj.io/trace/4764d8049dff52ea FAIL (2025-11-21 11:04:41 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 5.000792026s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 306.812316ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3d967bc11165dc1c FAIL (2025-11-21 11:04:31 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": context deadline exceeded Duration: 306.830861ms core website
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 428.464844ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace: https://jaeger.dev.storj.io/trace/3e5c193367f9a974 FAIL (2025-11-21 11:04:28 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 428.569503ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 2m0.001446739s remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace: https://jaeger.dev.storj.io/trace/3e5c193367f9a974 FAIL (2025-11-21 11:04:27 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": context deadline exceeded Duration: 2m0.001478028s edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 477.956264ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/e2bae25cf885024 FAIL (2025-11-21 11:04:26 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 477.971823ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 2m0.010764943s remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt trace: https://jaeger.dev.storj.io/trace/e2bae25cf885024 FAIL (2025-11-21 11:04:25 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 2m0.010787957s edge linksharing bgp eu1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 450.450988ms remote: "tcp:207.211.208.129:443" target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace: https://jaeger.dev.storj.io/trace/5285054b3a776da7 FAIL (2025-11-21 11:04:22 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318": context deadline exceeded Duration: 450.46727ms edge gateway vhost bgp eu1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 779.06083ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3d967bc11165dc1c FAIL (2025-11-21 11:04:16 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": context deadline exceeded Duration: 779.127467ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 2m0.000993127s remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3d967bc11165dc1c FAIL (2025-11-21 11:04:15 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": context deadline exceeded Duration: 2m0.001020751s core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.........................................................XXX] duration: 5.001340632s trace: https://jaeger.dev.storj.io/trace/1a094f628ff2a9e7 FAIL (2025-11-21 11:04:03 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 5.001370529s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........................................................XX] duration: 5.000639307s trace: https://jaeger.dev.storj.io/trace/6382621c84999325 FAIL (2025-11-21 11:03:09 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 5.000664906s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.001710668s trace: https://jaeger.dev.storj.io/trace/19eed30bfb4d73f1 FAIL (2025-11-21 11:02:37 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 5.00172778s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.804138453s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:27:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.804169291s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.774791637s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:26:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.774823626s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.301479677s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:25:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.301510176s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.858444049s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:24:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.858472644s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.839877337s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:24:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.83990536s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.919447896s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:23:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.919718072s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.922366566s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:22:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.92240015s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.714776615s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:21:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.71483231s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.762280761s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:20:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.762304275s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.787112188s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:19:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.787156643s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.727640089s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:19:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.727737043s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.797017468s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:18:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.797048628s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.831867821s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:18:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.831906606s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.83936698s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:17:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.839400192s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.774337574s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:16:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.774368362s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.828202194s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:15:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.828236369s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.26624425s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:14:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 6.26627028s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.786696532s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:13:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.786732459s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.810506391s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:13:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.810540064s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.858510081s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:12:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.858558102s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.85440926s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:12:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.854439336s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.487994947s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:11:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.488028301s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.839864174s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:10:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.839897478s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.700014653s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:09:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.700061551s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.688277512s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:08:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.688309715s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.32788494s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:08:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 6.327918725s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.478681329s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:07:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.47870784s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.450583612s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:06:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 6.450632455s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.75258192s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:05:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.752611386s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.339063774s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:05:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 6.339098098s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.47997217s trace: https://jaeger.dev.storj.io/trace/12edad1c937bf1cc FAIL (2025-11-21 08:04:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 21) ======LOGS====== ZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.928796022 (287h39m22.480266493s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.929322194 (287h39m22.480792655s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930182652 (287h39m22.481653113s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.930505397 (287h39m22.481975858s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.931427192 (287h39m22.482897653s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.936127943 (287h39m22.487598414s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:23.945702235 (287h39m22.497172697s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Uploading piece... Nov 21 07:43:24.777680580 (287h39m23.329151041s) [upload=253321]: Next returned nil segment Nov 21 07:43:24.777699406 (287h39m23.329169857s) [upload=253321]: Waiting for error group managing segments... Nov 21 07:43:25.167543642 (287h39m23.719014103s) [noise=true , address=185.167.204.206:4048 , piece_id=36XU7JMVGX7M4WCZBSKD5L6XJ53RO3F6G3KTH4EOG724RJQXAMRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171114011 (287h39m23.722584472s) [noise=true , address=185.167.204.206:4064 , piece_id=PACOKZPNPXQLK4Q2IH2EA6VTN4Y242J7LPMLQK2AY34TFJ66372A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171577493 (287h39m23.723047954s) [noise=true , address=185.167.204.205:4025 , piece_id=36ZUA5NHOBGNPWYWWWET6DVSAEIYSZLSEOS7AYUJJNGM3LBRUBPA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.171807842 (287h39m23.723278303s) [noise=true , address=185.167.204.206:4057 , piece_id=T63VVPXWMM63IU6EXOI343UERD3S6X3SDOM6KAKLCUAKQR3I6YRA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.173550250 (287h39m23.725020711s) [noise=true , address=185.167.204.206:4039 , piece_id=BO5IBEBDYDTKH5PWDAKI2SYWUZC7WISPRHIFV7V2N2WMWPTL2B2Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174015806 (287h39m23.725486257s) [noise=true , address=185.167.204.206:4044 , piece_id=DFEKPFZUOFJN2FHOSD5YQUJYE7D22KK43RZO4T35A6GN4YFN3MAA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.174683969 (287h39m23.72615442s) [noise=true , address=185.167.204.205:4024 , piece_id=7OZOKMPFW25H3W7JKSZQXBABATX6D2PSSFBW2IESGQV42CEVFJVA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.175380515 (287h39m23.726850976s) [noise=true , address=185.167.204.206:4053 , piece_id=MBKXDLH5CPAHNCE4HZBUXYPSF6HSXWA2JFF4ZACKDF2Q4DMM4ESA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.176349641 (287h39m23.727820102s) [noise=true , address=185.167.204.205:4010 , piece_id=344BOTQPEQ6RBHUSM4S7DCOVBCU7W6SYXHDOXUB5SGWFNZVE76NA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.183392662 (287h39m23.734863123s) [noise=true , address=185.167.204.205:4028 , piece_id=QLNUUEXDU4IOK3VLCID6FMU2RVA6OXXN5O3TJY6TCIV2KMCXEYSQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.696361567 (287h39m24.247832028s) [noise=true , address=81.171.14.70:4032 , piece_id=EHMYGDLZE23KNWUYCCMXSTR5MXADAQT6ZL2RYGNBTYM6HSGDJW3A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.715503287 (287h39m24.266973748s) [noise=true , address=81.171.14.70:4044 , piece_id=CL2LRAF7DMR2YFX2VMHDD6KFN2YZ4GDTMPM7CECFO7SZDJXQDIVQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.764827602 (287h39m24.316298063s) [noise=true , address=81.171.14.70:4046 , piece_id=SXCLHJI3ZS2FMKFZHIE6HCSBY7DHTATALP3P6ZRGUF7JFRQXXBNQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.792835015 (287h39m24.344305476s) [noise=true , address=81.171.14.70:4027 , piece_id=275SN3JYUEYOS5EONNB4D6UETKUW52ISUCX4KRAQ3ESP2P4SKWBQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.816863150 (287h39m24.368333611s) [noise=true , address=81.171.14.70:4035 , piece_id=QOL64VCOJRIVPIH6W37F2J7ZXA6ZWJJBOV75XYM7VFHSLHXVZ43A, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.817638266 (287h39m24.369108727s) [noise=true , address=81.171.14.70:4043 , piece_id=HEWHVMFZHEACNMGLLYBYNAPPOKWHNOWVMXXV3HZHV4DAHERQFWXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.835229685 (287h39m24.386700146s) [noise=true , address=81.171.14.70:4028 , piece_id=7Z2OFEKXMTFDS5HWEJEUS3OVOUEANUS7R3ZMDSBPYQZNPAULB7BQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:25.883706075 (287h39m24.435176537s) [noise=true , address=81.171.14.70:4029 , piece_id=EBNB3NUVKFF4UJMZZHPQVLCRBFXZ7T4S6S4Q25BB2PIRH577A7DA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.174320345 (287h39m24.725790806s) [noise=true , address=81.171.14.70:4025 , piece_id=LB6PTZNYHF7X4XM2YWYN24PGQLFAN5H5R6XGRRMLEUPYYKYCD5RA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.214432217 (287h39m24.765902668s) [noise=true , address=81.171.14.70:4031 , piece_id=63VLXJ6J3PVOD3XGXMD26DFGN4RHSC6ELTGQNHDYW4YYEQDXB5WA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:43:26.269654945 (287h39m24.821125416s) [noise=true , address=81.171.14.70:4048 , piece_id=74Y4MDP6PZ5UMTRVUGZLIV675Y37TPDX65GZWWM22UZYU6COWQUA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: <nil> Nov 21 07:44:23.215634392 (287h40m21.767104863s) [noise=true , address=89.149.201.147:4010 , piece_id=TQ2KTOWVJ5IRKODGEWPZTP6I3D5KFFFOHIXY4OIUFUDKHQB4MEYQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1P9roAnE3zK4HaPFarGVAb8L3EfByGr22ufkLcGUKBeF9SqBA9, address:89.149.201.147:4010): protocol: rpc: dial tcp 89.149.201.147:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216487847 (287h40m21.767958318s) [noise=true , address=89.149.201.147:4021 , piece_id=UJG4XL7NAWNZUSCCFVY7Y67EX6XBQZYHZ3KHRSTXQVX2JNF55BXA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:121wRVh3zqETdzR1KQbRUBmnd11SsxFkFHngWG8uxyndwsbUXJz, address:89.149.201.147:4021): protocol: rpc: dial tcp 89.149.201.147:4021: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216500130 (287h40m21.767970591s) [noise=true , address=89.149.201.147:4004 , piece_id=OFXDB4OUGACDHFPYSX4EUXMUPVM4I67QKBF24EOHZA3MJX6W6IHQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1u3qrydva66a46yxwwL8KhP22RKxyWH321urJzP6wn7tCiDmhr, address:89.149.201.147:4004): protocol: rpc: dial tcp 89.149.201.147:4004: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216512254 (287h40m21.767982705s) [noise=true , address=89.149.201.147:4012 , piece_id=VBB6RBGWHY7XXIKK6NQ376HJS6ESHFK54DJDH74BFOSGZDKGUPMQ, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv, address:89.149.201.147:4012): protocol: rpc: dial tcp 89.149.201.147:4012: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216525048 (287h40m21.767995509s) [noise=true , address=89.149.201.147:4015 , piece_id=AXMETGQA76RNGQ4MPZD6ODJ7ZN2YHP62GYISP7LKTMAT7NO5TM7Q, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12RbD6M2MHJfxcJXogZG8nUGFmeZjwUQ1auN6KGWJMPTffdK42K, address:89.149.201.147:4015): protocol: write tcp 188.166.225.170:37564->89.149.201.147:4015: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216535077 (287h40m21.768005538s) [noise=true , address=89.149.201.147:4014 , piece_id=RE5PEQ53C22EMOOZH4THPQZOQHI2CK42NOKY3FYWDF3TFI5XJPJA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr, address:89.149.201.147:4014): protocol: read tcp 188.166.225.170:45644->89.149.201.147:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216544906 (287h40m21.768015367s) [noise=true , address=89.149.201.147:4011 , piece_id=JTJOO26SDW7IFNXONCLYYA7RZE2VAMG7D6TDGOVCFPURIPAZBBOA, seg_pos={0 0}, upload=253321]: Done uploading piece. err: ecclient: upload failed (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg, address:89.149.201.147:4011): protocol: read tcp 188.166.225.170:48630->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 21 07:44:23.216573700 (287h40m21.768044151s) [seg_pos={0 0}, upload=253321]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil> Nov 21 07:44:23.389269985 (287h40m21.940740446s) [upload=253321]: Done uploading segments... Nov 21 07:44:23.389315001 (287h40m21.940785462s) [upload=253321]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21) Duration: 5.480005304s core uplink eu-select-1 eu1-placement-15

Orbiter Check History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)