Orbiter History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [......X.X......] duration: 882.769388ms remote: "tcp:3.109.243.18:443" response headers: Date=Thu, 23 Oct 2025 17:40:42 GMT Content-Type=text/html Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red target: https://www.storj.io/dcs-satellites FAIL (2025-10-23 17:45:21 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 882.797812ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [.......X.X.....] duration: 591.910832ms remote: "tcp:3.109.243.18:443" response headers: Date=Thu, 23 Oct 2025 17:40:42 GMT Content-Type=text/html Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red target: https://www.storj.io/dcs-satellites FAIL (2025-10-23 17:44:11 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 591.943004ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [........X.X....] duration: 860.277338ms remote: "tcp:3.109.243.18:443" response headers: Date=Thu, 23 Oct 2025 17:40:42 GMT Content-Type=text/html Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red target: https://www.storj.io/dcs-satellites FAIL (2025-10-23 17:43:24 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 860.306684ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [.........X.X...] duration: 1.171446724s remote: "tcp:3.109.243.18:443" response headers: Date=Thu, 23 Oct 2025 17:40:42 GMT Content-Type=text/html Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red target: https://www.storj.io/dcs-satellites FAIL (2025-10-23 17:42:52 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 1.171480337s core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [..........X.X..] duration: 625.428605ms remote: "tcp:3.109.243.18:443" response headers: Date=Thu, 23 Oct 2025 17:40:42 GMT Content-Type=text/html Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red target: https://www.storj.io/dcs-satellites FAIL (2025-10-23 17:42:00 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 625.456559ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [...........X.X.] duration: 652.798433ms remote: "tcp:3.109.243.18:443" response headers: Date=Thu, 23 Oct 2025 17:40:42 GMT Content-Type=text/html Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red target: https://www.storj.io/dcs-satellites FAIL (2025-10-23 17:41:00 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 652.824423ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [............X.X] duration: 20.251242916s remote: "tcp:3.109.243.18:443" response headers: Date=Thu, 23 Oct 2025 17:40:42 GMT Content-Type=text/html Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red target: https://www.storj.io/dcs-satellites FAIL (2025-10-23 17:40:42 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 20.251282872s core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 8.020010301s remote: "tcp:3.109.243.18:443" response headers: Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red Date=Thu, 23 Oct 2025 17:39:49 GMT Content-Type=text/html target: https://www.storj.io/dcs-satellites FAIL (2025-10-23 17:39:58 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 8.020030328s core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 20.251171586s remote: "tcp:3.109.243.18:443" response headers: Content-Length=164 X-Cluster-Name=ap-south-1-prod-hosting-red Date=Thu, 23 Oct 2025 17:39:49 GMT Content-Type=text/html target: https://www.storj.io/dcs-satellites FAIL (2025-10-23 17:39:49 UTC) orbiter warning: unexpected status code "504 Gateway Time-out" "<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Duration: 20.251198035s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 12.991041356s FAIL (2025-10-23 16:31:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 12.991068818s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.033064971s FAIL (2025-10-23 16:30:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.033094888s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.955873172s FAIL (2025-10-23 16:29:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 5.955904131s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.062974903s FAIL (2025-10-23 16:28:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.063006263s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.06343103s FAIL (2025-10-23 16:28:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.063522644s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.010556777s FAIL (2025-10-23 16:27:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.010587906s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.086473547s FAIL (2025-10-23 16:26:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.086504385s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.044223123s FAIL (2025-10-23 16:26:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.044258169s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.999291238s FAIL (2025-10-23 16:25:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 5.999329641s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.32165308s FAIL (2025-10-23 16:24:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.321684179s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.041894411s FAIL (2025-10-23 16:23:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.041929678s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.079334262s FAIL (2025-10-23 16:22:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.079365171s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.176008988s FAIL (2025-10-23 16:21:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.176045798s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.133680918s FAIL (2025-10-23 16:20:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.13370786s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.054508961s FAIL (2025-10-23 16:20:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.054546994s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.017450825s FAIL (2025-10-23 16:19:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.017482345s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.124807348s FAIL (2025-10-23 16:19:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.124842025s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.348958022s FAIL (2025-10-23 16:18:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.348983181s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 25.031249253s FAIL (2025-10-23 16:18:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 25.031275031s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.079835676s FAIL (2025-10-23 16:17:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.079866255s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.008246561s FAIL (2025-10-23 16:16:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.008272982s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.992616221s FAIL (2025-10-23 16:16:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 5.992650827s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.986748386s FAIL (2025-10-23 16:15:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 5.986780268s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.945310574s FAIL (2025-10-23 16:15:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 5.945352554s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.094415892s FAIL (2025-10-23 16:14:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.094439095s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.672440381s FAIL (2025-10-23 16:13:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.672469926s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.890445793s FAIL (2025-10-23 16:12:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 5.890475861s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.12266857s FAIL (2025-10-23 16:12:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.122697334s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.898238319s FAIL (2025-10-23 16:11:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 5.898338361s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.870955368s FAIL (2025-10-23 16:10:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 5.871000805s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.992854544s FAIL (2025-10-23 16:09:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 5.992883088s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.096544754s FAIL (2025-10-23 16:09:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.096581724s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.867359764s FAIL (2025-10-23 16:07:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 5.86745231s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X................................X..........................] duration: 6.006733701s FAIL (2025-10-23 16:07:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.006989349s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X................................X.........................] duration: 6.140732047s FAIL (2025-10-23 16:05:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.140766151s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X................................X........................] duration: 10.542497731s FAIL (2025-10-23 16:04:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 10.542531315s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X................................X.......................] duration: 6.148427939s FAIL (2025-10-23 16:04:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.148497501s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X................................X......................] duration: 5.984960074s FAIL (2025-10-23 16:03:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 5.984996614s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X................................X.....................] duration: 6.012227518s FAIL (2025-10-23 16:02:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.012299685s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X................................X....................] duration: 5.937001697s FAIL (2025-10-23 16:01:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 5.937039387s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X................................X...................] duration: 6.075174281s FAIL (2025-10-23 16:01:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ) context deadline exceeded; piecestore: (Node ID: 1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, Piece ID: FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA) context deadline exceeded ======LOGS====== Oct 23 15:47:30.574514773 (722h16m16.036607246s) [upload=635353]: Starting upload Oct 23 15:47:30.574599916 (722h16m16.036692399s) [upload=635353]: Done starting upload Oct 23 15:47:30.574689115 (722h16m16.036781598s) [upload=635353]: Uploading segments... Oct 23 15:47:30.574825665 (722h16m16.036918148s) [upload=635353]: Got next segment. Inline: false Oct 23 15:47:30.574872956 (722h16m16.036965439s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 23 15:47:30.574891551 (722h16m16.036984024s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 23 15:47:30.821098787 (722h16m16.28319128s) [seg_pos={0 0}, upload=635353]: Begin upload segment... Oct 23 15:47:30.821159984 (722h16m16.283252467s) [seg_pos={0 0}, upload=635353]: Done begin upload segment. Oct 23 15:47:30.821499770 (722h16m16.283592253s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.821581386 (722h16m16.283673869s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.827115314 (722h16m16.289207797s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829037584 (722h16m16.291130077s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.829406918 (722h16m16.291499391s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.830827553 (722h16m16.292920036s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.831014569 (722h16m16.293107052s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832391681 (722h16m16.294484155s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.832830597 (722h16m16.29492308s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.833989404 (722h16m16.296081887s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.834620115 (722h16m16.296712598s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.835846700 (722h16m16.297939183s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.836613550 (722h16m16.298706033s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.837468389 (722h16m16.299560872s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.838767191 (722h16m16.300859685s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:30.839371111 (722h16m16.301463584s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Uploading piece... Oct 23 15:47:32.459327450 (722h16m17.921419933s) [upload=635353]: Next returned nil segment Oct 23 15:47:32.459350074 (722h16m17.921442547s) [upload=635353]: Waiting for error group managing segments... Oct 23 15:47:32.844834471 (722h16m18.306926954s) [noise=true , address=145.79.227.21:4474 , piece_id=IZWQ5OTECWYE44H5EUH4DSBG4FNLNPM4I5RJCHZQBD3LNTY3LRMA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855161334 (722h16m18.317253807s) [noise=true , address=145.79.227.20:4438 , piece_id=4A2RBRVF2DTRRPN4YUYIRFU5U2PLVBMKVSSFFBCXCSPB3KZ6P7EA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.855210178 (722h16m18.317302651s) [noise=true , address=145.79.227.14:4838 , piece_id=QC6SJVNZVL6Y4JQ36HLHPFWSDATJWS3EXNCQBSQC63GXCCRGTKGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.859252124 (722h16m18.321344598s) [noise=true , address=145.79.227.13:4802 , piece_id=5MKENTITMZDIZXUEM2LCRGYB5SL2P6BQCY4KHHQYZGV25HB23JOA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.863373742 (722h16m18.325466225s) [noise=true , address=145.79.227.18:4007 , piece_id=UMPKX4KJPXQOSHOHGD724QH5W6IJPMMK5VINBPGKJ6Y4M3QSZ6XA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.867982659 (722h16m18.330075142s) [noise=true , address=145.79.227.19:4077 , piece_id=2FCJEHKPIY3K3A6WPHULSPNGCVCAUXVAZK27TTSJWPAQ4QNLZYEA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.950589769 (722h16m18.412682252s) [noise=true , address=145.79.227.26:4655 , piece_id=OZMOWLSVFMJ2TNMTVYEFZ5DMY3YZV7YNAV6JHXUD5MLRC63F6JGQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.955456275 (722h16m18.417548748s) [noise=true , address=145.79.227.4:4185 , piece_id=ZGT5LX5YJX6HSQN5NTHR4DECEHE4UNPU7JDRWGHFBERVTBOJEQFA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.960006550 (722h16m18.422099033s) [noise=true , address=145.79.227.12:4767 , piece_id=QCT6TY364GS4VT66QSAZPKAEHSNWK5TWDSQOOWLXA6G6AMRC3T4Q, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:32.970320610 (722h16m18.432413083s) [noise=true , address=145.79.227.17:4972 , piece_id=X2JDVRCLUIP3MED5KM3KET4AGJ5KJ464QUREJSJNZD265KP37HZQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.043464029 (722h16m18.505556512s) [noise=true , address=45.91.116.10:4082 , piece_id=DMQ2COKPRWVG2VSRKKIE22AJI7AD6RIAOSVPDHUCPH26URYVX5LQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.059123358 (722h16m18.521215831s) [noise=true , address=45.91.116.7:4009 , piece_id=FKMB336W76GPPJI2PQSG6PAOYEZW5HLK6CEKKWUX6MCNUQZUXEBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097309502 (722h16m18.559401975s) [noise=true , address=77.90.31.21:4879 , piece_id=NGOSEGQUVLZIEKW6MFU45YGVADBT23NWR6I7HAC7HGJ7VRGDRMFQ, seg_pos={0 0}, upload=635353]: Done uploading piece. err: <nil> Oct 23 15:47:33.097355009 (722h16m18.559447482s) [seg_pos={0 0}, upload=635353]: Segment reached optimal threshold of 13 pieces. Oct 23 15:47:33.097496818 (722h16m18.559589292s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097503962 (722h16m18.559596445s) [noise=true , address=77.90.31.10:4520 , piece_id=YN4VDRNMRXGLXMAQ5O3YQNLAF5WVSWZSC7A45N3ME3HCBHONRCBA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097557404 (722h16m18.559649887s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12P5cnEza1Xxwg4qo1nau11zYHsHxqEAwhFsnt2UE1RhtTrJmra): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097561862 (722h16m18.559654336s) [noise=true , address=145.79.227.23:4545 , piece_id=OJKAODFPT2IE74SETAXNNEUTIMJYQG73UMEGTRESNH7U3XJEGOTA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097686750 (722h16m18.559779233s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: expected piece hash; context canceled; EOF Oct 23 15:47:33.097708361 (722h16m18.559800844s) [noise=true , address=77.90.31.15:4699 , piece_id=TO3LZ5YDIQTAOZEF6KM4NN7F5ELZU7XILHQW6OSSEI2N6TTH24TA, seg_pos={0 0}, upload=635353]: Long tail cancellation due to optimal threshold Oct 23 15:47:33.097756803 (722h16m18.559849276s) [seg_pos={0 0}, upload=635353]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 23 15:47:33.097883605 (722h16m18.559976098s) [upload=635353]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 23 15:47:33.097890307 (722h16m18.559982791s) [upload=635353]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 23 15:47:33.391188634 (722h16m18.853281117s) [upload=635353]: Done uploading segments... Oct 23 15:47:33.391212339 (722h16m18.853304822s) [upload=635353]: Upload finished. err: <nil> Duration: 6.075210599s core uplink us-select-1

Orbiter Check History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)