Orbiter History

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 7.656168428s FAIL (2025-10-04 09:57:05 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout Duration: 7.656207834s edge authservice http bgp ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 37.224011ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 FAIL (2025-10-04 09:57:03 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp 207.211.208.129:443: i/o timeout Duration: 37.277042ms edge gateway vhost bgp ap1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 902.840164ms FAIL (2025-10-04 09:56:49 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout Duration: 902.874739ms edge authservice http bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [........................................................X.X.] duration: 171.128134ms target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:56:44 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 171.161446ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 265.775562ms target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:56:41 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 265.802582ms edge linksharing bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.568735712s FAIL (2025-10-04 09:56:40 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout Duration: 5.568750349s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 39.486951ms target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:56:37 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 39.528449ms edge linksharing bgp ap1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 449.493208ms FAIL (2025-10-04 09:56:35 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout Duration: 449.529097ms edge authservice http bgp ap1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 20.959881277s FAIL (2025-10-04 09:56:34 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout Duration: 20.959905514s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [.........................................................X.X] duration: 30.000666595s target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:56:33 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 30.000698896s edge linksharing bgp eu1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001223899s FAIL (2025-10-04 09:56:28 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout Duration: 30.001257764s edge authservice http bgp eu1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001090409s FAIL (2025-10-04 09:56:12 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout Duration: 30.001124893s edge authservice http bgp ap1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 30.918085ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 FAIL (2025-10-04 09:56:00 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011": dial tcp 207.211.208.129:443: i/o timeout Duration: 30.95765ms edge gateway vhost bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 29.452413ms target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:55:54 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 29.477762ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 612.085067ms remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:55:47 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 612.125584ms edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 2m0.00043084s remote: "tcp:207.211.208.129:443" target: https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:55:46 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded Duration: 2m0.000462821s edge linksharing bgp eu1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 261.71546ms target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:55:45 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 261.747331ms edge linksharing bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.745236593s FAIL (2025-10-04 09:55:09 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout Duration: 5.745264276s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 25.825386ms target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:54:58 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 25.84782ms edge linksharing bgp ap1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001395365s FAIL (2025-10-04 09:54:52 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 207.211.208.129:443: i/o timeout Duration: 30.001423449s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 274.032404ms target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:54:47 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 274.06736ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 24.869565ms target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:53:44 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 24.896687ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 264.385635ms target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:53:33 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 264.418046ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 276.196085ms target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:53:16 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 276.214229ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 23.496933ms target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:52:49 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 23.52667ms edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 30.001410255s target: https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:52:45 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 30.001446484s edge linksharing bgp ap1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 260.178513ms target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:51:31 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 260.20373ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 30.000810751s target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-10-04 09:51:30 UTC) orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": dial tcp 207.211.208.129:443: i/o timeout Duration: 30.000894721s edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.175802438s FAIL (2025-10-04 09:31:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.175864015s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 5.883023575s FAIL (2025-10-04 09:31:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.883062921s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 6.072017658s FAIL (2025-10-04 09:30:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.072050832s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 5.955947433s FAIL (2025-10-04 09:29:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.955987891s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 5.666688094s FAIL (2025-10-04 09:28:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.66672231s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 6.133787261s FAIL (2025-10-04 09:28:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.133815075s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 6.030780005s FAIL (2025-10-04 09:27:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.030817867s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 5.89967807s FAIL (2025-10-04 09:26:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.899701043s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 6.02971779s FAIL (2025-10-04 09:25:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.029759581s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 9.419405205s FAIL (2025-10-04 09:25:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 9.419465018s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 5.767446922s FAIL (2025-10-04 09:23:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.767478853s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 5.813294858s FAIL (2025-10-04 09:23:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.813326127s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 6.072889194s FAIL (2025-10-04 09:22:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.07291975s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 5.921778923s FAIL (2025-10-04 09:21:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 5.921811203s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 6.009832775s FAIL (2025-10-04 09:20:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.009862641s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 6.315375061s FAIL (2025-10-04 09:19:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.315408246s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 6.166100831s FAIL (2025-10-04 09:18:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.166164283s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 6.00582204s FAIL (2025-10-04 09:18:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.005881493s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 6.275854698s FAIL (2025-10-04 09:17:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.275878843s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 6.088314344s FAIL (2025-10-04 09:16:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.088340744s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 6.086055356s FAIL (2025-10-04 09:15:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.086089931s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 6.076586759s FAIL (2025-10-04 09:14:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, Piece ID: JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA) context deadline exceeded; piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ) context deadline exceeded ======LOGS====== 5:52.375988210 (259h14m37.838080693s) [upload=228151]: Starting upload Oct 4 08:45:52.376083322 (259h14m37.838175805s) [upload=228151]: Done starting upload Oct 4 08:45:52.376283834 (259h14m37.838376317s) [upload=228151]: Uploading segments... Oct 4 08:45:52.376349869 (259h14m37.838442342s) [upload=228151]: Got next segment. Inline: false Oct 4 08:45:52.376390216 (259h14m37.838482699s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 4 08:45:52.376396318 (259h14m37.838488801s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 4 08:45:52.625441732 (259h14m38.087534215s) [seg_pos={0 0}, upload=228151]: Begin upload segment... Oct 4 08:45:52.625515953 (259h14m38.087608436s) [seg_pos={0 0}, upload=228151]: Done begin upload segment. Oct 4 08:45:52.625553925 (259h14m38.087646408s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625566059 (259h14m38.087658542s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.625625000 (259h14m38.087717483s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.626094385 (259h14m38.088186878s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.627777880 (259h14m38.089870363s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.628492331 (259h14m38.090584814s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.629687286 (259h14m38.091779759s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.630128106 (259h14m38.092220589s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.632374063 (259h14m38.094466546s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.636122581 (259h14m38.098215074s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.639042832 (259h14m38.101135305s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.641984193 (259h14m38.104076676s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.642224862 (259h14m38.104317345s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644090234 (259h14m38.106182717s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.644989627 (259h14m38.1070821s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:52.647232698 (259h14m38.109325182s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Uploading piece... Oct 4 08:45:54.509252523 (259h14m39.971345007s) [upload=228151]: Next returned nil segment Oct 4 08:45:54.509273354 (259h14m39.971365837s) [upload=228151]: Waiting for error group managing segments... Oct 4 08:45:54.800724081 (259h14m40.262816554s) [noise=true , address=45.91.116.14:4190 , piece_id=LUMBBNT4376D3JA5NV7LGGDLYRT7YRXG2NWQSNVFACM2KHSILIDQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:54.846553803 (259h14m40.308646286s) [noise=true , address=45.91.116.8:4048 , piece_id=6755GX3RU7ZPEQE6TFC3CKOI3ZPSSA3RDQ227QYBVGF5W436QFHQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.021453312 (259h14m40.483545795s) [noise=true , address=45.91.116.12:4227 , piece_id=YOJPNSBXUCETTLJKDP3CC5ZU32GABCLIYTQCYOUYOONBPDFFD6IA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.464323145 (259h14m40.926415629s) [noise=true , address=145.79.227.2:4113 , piece_id=DTYLUJCYZ45NU52N5B73SFWENRT27LJLXBU6IDTCKMUZLC5VT3YA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.475183536 (259h14m40.937276009s) [noise=true , address=145.79.227.14:4839 , piece_id=NNRCXQLPARJ6RONKT6HLKYLMSFW6E62SO4Q3IRMW4BVDPIJU3SHA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:55.952385858 (259h14m41.414478331s) [noise=true , address=145.79.227.18:4007 , piece_id=EPU4WFY3GIAND2THT2VVLY4WA6NCO4JILPKCHTCJ7BXICY56WQGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.234208460 (259h14m41.696300943s) [noise=true , address=145.79.227.11:4729 , piece_id=JK5RO7PCAM3DVHGY5BNJEPHPQRKY2ZTIKCE6JIQ6WJOZZHYO44PA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:56.562151298 (259h14m42.024243771s) [noise=true , address=145.79.227.7:4294 , piece_id=K6GNFJGGJ5VOEUDXDHRLZ3EDMMCN4IAL7MOBZ4RSWDMDY4E5Y2OA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:57.073680377 (259h14m42.535772851s) [noise=true , address=145.79.227.15:4876 , piece_id=HJJOUHFY3FVAZM4OBLGNCBUFY4XX5SO3VVFRKVIGORM6MAYVGPNQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.183381119 (259h14m43.645473612s) [noise=true , address=145.79.227.12:4767 , piece_id=D64TUK6ZD5A3N6JKGIBNPEXRANGD7SFBLQSCNKD6NLO5PRQ2VYMA, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:58.608113053 (259h14m44.070205536s) [noise=true , address=145.79.227.6:4257 , piece_id=4FDA7ZVOKTQJGS5TYKTEXRXWIVHEBWSXCV7RB2LKGHJ66LTR6CTQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:45:59.842625061 (259h14m45.304717544s) [noise=true , address=145.79.227.4:4185 , piece_id=72J2DHSMMKOPZ34PPZAOSHXYPN7Y7JFMAJZU3YHRE2V2TQ7HPUWQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105160734 (259h14m45.567253217s) [noise=true , address=145.79.227.13:4804 , piece_id=FTKSXBZQ6F5K7HQ23EGUJ53JVPS272PKD3ZQWGRKEAXWS5BSXHJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: <nil> Oct 4 08:46:00.105196673 (259h14m45.567289156s) [seg_pos={0 0}, upload=228151]: Segment reached optimal threshold of 13 pieces. Oct 4 08:46:00.105358561 (259h14m45.567451044s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105366125 (259h14m45.567458608s) [noise=true , address=145.79.227.16:4911 , piece_id=U2J2CZLH37W5EBZXLZJDFZ6KGE5J4DHP66LDX2NHF667DYAUCHGQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105481937 (259h14m45.56757442s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49460->145.79.227.17:4971: use of closed network connection Oct 4 08:46:00.105488559 (259h14m45.567581032s) [noise=true , address=145.79.227.17:4971 , piece_id=JXCKFWMYCJWYRJHMHHGL4WZTBNOWZVM5ENOIGEVZUW2IBJEXWUJQ, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105600943 (259h14m45.567693426s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE): protocol: expected piece hash; context canceled; EOF Oct 4 08:46:00.105631541 (259h14m45.567724024s) [noise=true , address=145.79.227.5:4223 , piece_id=6PVXAFP74NPMJONFGBDOO5ERZGLO4ZYGAHKZUQTLC2NRDJHH4Y6Q, seg_pos={0 0}, upload=228151]: Long tail cancellation due to optimal threshold Oct 4 08:46:00.105669263 (259h14m45.567761746s) [seg_pos={0 0}, upload=228151]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 4 08:46:00.105858804 (259h14m45.567951297s) [upload=228151]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 4 08:46:00.105869715 (259h14m45.567962198s) [upload=228151]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 4 08:46:00.441371898 (259h14m45.903464391s) [upload=228151]: Done uploading segments... Oct 4 08:46:00.441398980 (259h14m45.903491453s) [upload=228151]: Upload finished. err: <nil> Duration: 6.076628227s core uplink us-select-1

Orbiter Check History

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)

checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)