Orbiter History

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 900.777797ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:49:29 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 900.797506ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 945.426177ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:48:57 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 945.450122ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 930.78562ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:48:00 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 930.805868ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 921.121307ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:47:31 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 921.150954ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 888.482357ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:46:55 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 888.50392ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 911.067501ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:45:56 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 911.08738ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 875.100054ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:45:03 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 875.152975ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.584471904s remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:43:57 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 2.584496552s edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 895.327965ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:42:58 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 895.346972ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 884.297099ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:42:06 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 884.321737ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 962.906001ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:41:08 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 962.933815ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 912.540188ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:40:03 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 912.56722ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 906.477367ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:39:29 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 906.505712ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 918.220064ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:38:16 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 918.247086ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 906.334223ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:37:38 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 906.346567ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 893.51696ms remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:37:20 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 893.535907ms edge ipfs us1
checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1.046006426s remote: "tcp:5.161.37.179:443" response headers: Access-Control-Allow-Origin=* Content-Length=0 Date=Thu, 17 Jul 2025 00:36:44 GMT target: https://www.storj-ipfs.com/api/v0/add FAIL (2025-07-17 00:36:44 UTC) orbiter warning: unexpected status code "502 Bad Gateway" "" Duration: 1.046029601s edge ipfs us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.910108341s FAIL (2025-07-16 19:47:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.910137355s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.678723483s FAIL (2025-07-16 19:47:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.678749263s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X.X.........................................................] duration: 5.766476273s FAIL (2025-07-16 19:45:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.766506221s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X.X........................................................] duration: 5.659813944s FAIL (2025-07-16 19:45:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.659841237s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X.X.......................................................] duration: 6.80273274s FAIL (2025-07-16 19:44:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.802761226s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X.X......................................................] duration: 5.722388212s FAIL (2025-07-16 19:43:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.722413972s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X.X.....................................................] duration: 5.974239375s FAIL (2025-07-16 19:42:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.974259453s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X.X....................................................] duration: 5.870835398s FAIL (2025-07-16 19:42:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.870864592s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X.X...................................................] duration: 5.885496191s FAIL (2025-07-16 19:41:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.885519808s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X.X..................................................] duration: 5.933418137s FAIL (2025-07-16 19:41:14 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.933439487s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X.X.................................................] duration: 5.739433316s FAIL (2025-07-16 19:40:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.739457992s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X.X................................................] duration: 5.960390857s FAIL (2025-07-16 19:39:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.960417568s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X.X...............................................] duration: 5.760795005s FAIL (2025-07-16 19:38:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.760818379s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X.X..............................................] duration: 5.982737359s FAIL (2025-07-16 19:37:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.982764441s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X.X.............................................] duration: 5.671895735s FAIL (2025-07-16 19:36:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.671917296s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.335946842s FAIL (2025-07-16 19:36:16 UTC) orbiter warning: context deadline exceeded Duration: 6.335959234s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X.X............................................] duration: 5.880492351s FAIL (2025-07-16 19:35:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.880516507s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.183023655s FAIL (2025-07-16 19:35:11 UTC) orbiter warning: context deadline exceeded Duration: 6.183049174s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.265048878s FAIL (2025-07-16 19:34:49 UTC) orbiter warning: context deadline exceeded Duration: 6.265067062s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X.X...........................................] duration: 5.93055172s FAIL (2025-07-16 19:34:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.930579413s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.116277037s FAIL (2025-07-16 19:33:37 UTC) orbiter warning: context deadline exceeded Duration: 6.11629988s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X.X..........................................] duration: 5.996792491s FAIL (2025-07-16 19:33:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.996819171s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.304465205s FAIL (2025-07-16 19:33:01 UTC) orbiter warning: context deadline exceeded Duration: 6.304480505s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X.X.........................................] duration: 5.886203468s FAIL (2025-07-16 19:32:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.886223988s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.058531989s FAIL (2025-07-16 19:31:46 UTC) orbiter warning: context deadline exceeded Duration: 6.058561917s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.542301665s FAIL (2025-07-16 19:31:31 UTC) orbiter warning: context deadline exceeded Duration: 6.542320431s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X.X........................................] duration: 5.947793098s FAIL (2025-07-16 19:31:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.9478199s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X.X.......................................] duration: 5.821751628s FAIL (2025-07-16 19:31:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.821775362s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.229759486s FAIL (2025-07-16 19:30:37 UTC) orbiter warning: context deadline exceeded Duration: 6.229781239s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................X.X......................................] duration: 6.022316832s FAIL (2025-07-16 19:30:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.022345957s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................X.X.....................................] duration: 5.828156364s FAIL (2025-07-16 19:29:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 16564 (561h53m24.798457899s) [upload=449313]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 16 19:03:50.027820431 (561h53m24.798461756s) [upload=449313]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 16 19:03:50.325177169 (561h53m25.095818494s) [upload=449313]: Done uploading segments... Jul 16 19:03:50.325193620 (561h53m25.095834955s) [upload=449313]: Upload finished. err: <nil> Jul 16 19:04:48.569836177 (561h54m23.340477512s) [upload=449326]: Starting upload Jul 16 19:04:48.569938393 (561h54m23.340579728s) [upload=449326]: Done starting upload Jul 16 19:04:48.570035328 (561h54m23.340676663s) [upload=449326]: Uploading segments... Jul 16 19:04:48.570178030 (561h54m23.340819366s) [upload=449326]: Got next segment. Inline: false Jul 16 19:04:48.570225461 (561h54m23.340866797s) [upload=449326]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 16 19:04:48.570660875 (561h54m23.34130221s) [upload=449326]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 16 19:04:48.802766945 (561h54m23.57340827s) [seg_pos={0 0}, upload=449326]: Begin upload segment... Jul 16 19:04:48.802809045 (561h54m23.57345038s) [seg_pos={0 0}, upload=449326]: Done begin upload segment. Jul 16 19:04:48.802829214 (561h54m23.573470549s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.802862217 (561h54m23.573503552s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803207267 (561h54m23.573848602s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803427409 (561h54m23.574068744s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803527329 (561h54m23.574168664s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803732292 (561h54m23.574373627s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.803830118 (561h54m23.574471453s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804030362 (561h54m23.574671697s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804134861 (561h54m23.574776196s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804411341 (561h54m23.575052676s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804626612 (561h54m23.575267947s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.804816937 (561h54m23.575458272s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805045004 (561h54m23.575686339s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805130928 (561h54m23.575772263s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805372630 (561h54m23.576013965s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:48.805623861 (561h54m23.576265196s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Uploading piece... Jul 16 19:04:50.496612514 (561h54m25.267253849s) [upload=449326]: Next returned nil segment Jul 16 19:04:50.496630077 (561h54m25.267271413s) [upload=449326]: Waiting for error group managing segments... Jul 16 19:04:50.859098237 (561h54m25.629739572s) [noise=true , address=104.36.82.8:4972 , piece_id=L7CXAJ2DUQIWVUIHBPLUVL43BMAHJWRPX732DPO6NRVKUDJCKJDQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.863702978 (561h54m25.634344313s) [noise=true , address=45.91.116.7:4009 , piece_id=TVUNFPNS3VOUFQ32YTH2RBRL66SGMDXFEU5BGBDHAPZEMN5M5IMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864453323 (561h54m25.635094658s) [noise=true , address=45.91.116.13:4154 , piece_id=UIJ6G5XGBVWYUSQEV65UDX27GPF4NDB52J7WZQNPSQ2LRRNJOWCA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864756403 (561h54m25.635397738s) [noise=true , address=104.36.82.8:4970 , piece_id=6U62DE2ZJQ27HK7QXRHWDQAYQCSE3JHGTDM3SHX3JLFVB57FUXOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.864829954 (561h54m25.635471289s) [noise=true , address=104.36.82.6:4876 , piece_id=F4D2YJLR2677R5WN4W27AO7PVY3DMDV7PK7S4YGHUZ3FYUFTCUPQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.929638304 (561h54m25.700279639s) [noise=true , address=104.36.82.9:4007 , piece_id=COPN7CHQV4HZP74KELOGKBSMUIBKUIV74TI4I3I5H2R4UZ4E2QMQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.933494202 (561h54m25.704135547s) [noise=true , address=104.36.82.13:4185 , piece_id=VBKPYYDBOJ6YYJCH2F4VRO7ZI4M2FN2NQ2H74ZPFG5WWYNW2NUBQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:50.991985657 (561h54m25.762626992s) [noise=true , address=104.36.82.16:4295 , piece_id=AC3PK7BZKTL2NACWPYFIBWQ2TL3BYU7BKRJRMGVBNQD6HOBAF2JQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.007934743 (561h54m25.778576078s) [noise=true , address=104.36.82.14:4222 , piece_id=GPJV36JGX6DTYLF5ITKSB7BCBX4KUANV4RKV5LRIJKLZ33XBRIWQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.010915507 (561h54m25.781556832s) [noise=true , address=104.36.82.11:4113 , piece_id=O7S74GXF5AX6L5XLNSQLNKY2HMDEM452TCGTAHPJLDQRXOR5X7UQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:04:51.011671332 (561h54m25.782312658s) [noise=true , address=104.36.82.4:4801 , piece_id=HKYKV4OF5C6PLFHV3ZRPKDE5S3OYRRAAKK2UPPEVRINLSJDFIM6A, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.583107142 (561h54m52.353748477s) [noise=true , address=77.90.31.19:4841 , piece_id=32BPDUKYQRIIPMSNAG6KNM2QG5AFMCFQD5RZZWFIUMCCU52BR4QQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: <nil> Jul 16 19:05:17.926380154 (561h54m52.697021489s) [noise=true , address=77.90.31.15:4697 , piece_id=4S6F332OXLEBTNRTKKSLG2CFI47H4WQKTFX35OL7HYIVDVYOVXBA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: expected piece hash; context deadline exceeded; EOF Jul 16 19:05:17.926475696 (561h54m52.697117031s) [noise=true , address=77.90.31.19:4842 , piece_id=XPZNUBJORWYFO4CUYKVZR2CYDVOEVEFVPFUNGR5S554Y6VKYJ3OA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47984->77.90.31.19:4842: use of closed network connection Jul 16 19:05:17.926548556 (561h54m52.697189891s) [noise=true , address=77.90.31.18:4807 , piece_id=RWQPBP527NLG7LQKV2Q5KGCZHYPUOY64NCMWD3LZTKR4H5IB6HKA, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45628->77.90.31.18:4807: use of closed network connection Jul 16 19:05:17.926617057 (561h54m52.697258392s) [noise=true , address=77.90.31.12:4589 , piece_id=BPOO2PW5QE4O4PK25YONYV5FG257T6JNZH6BRD2GKAFLQVVNUMOQ, seg_pos={0 0}, upload=449326]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40108->77.90.31.12:4589: use of closed network connection Jul 16 19:05:17.926638227 (561h54m52.697279562s) [seg_pos={0 0}, upload=449326]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 16 19:05:18.156778165 (561h54m52.9274195s) [upload=449326]: Done uploading segments... Jul 16 19:05:18.156800027 (561h54m52.927441362s) [upload=449326]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.828179338s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.117877347s FAIL (2025-07-16 19:29:31 UTC) orbiter warning: context deadline exceeded Duration: 6.117899849s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.286233581s FAIL (2025-07-16 19:28:41 UTC) orbiter warning: context deadline exceeded Duration: 6.286250084s edge gateway us-select-1

Orbiter Check History

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.HTTP{"POST" "https://www.storj-ipfs.com/api/v0/add", authorization: <redacted>, ExpectedContent: "Qmbj74UUMUGDe4TES4fEbWEEe5fiwqJEjQ8CHaLJUF9dSi"} failure (antiblip window 3/60): (ipfs.post)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)