Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......X.........X.................X........................] duration: 1.841260994s FAIL (2025-07-26 21:51:35 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.841294889s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 11 out of 60 is failed (threshold=3) [..X...................................XX.........XX.XX.XXXX.] duration: 23.105978519s FAIL (2025-07-26 21:51:20 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 23.106s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 22 out of 60 is failed (threshold=3) [......................................XXXXXXXXXXXXXXXXXXXXXX] duration: 1m0.233648252s FAIL (2025-07-26 21:51:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _pos={0 0}, upload=55892]: Uploading piece... Jul 26 21:50:02.084083340 (58h14m11.80664639s) [noise=true , address=104.36.82.12:4169 , piece_id=XAHRR6PMD5EFWADGHASVA66R27PFCYR5A6XC7QON7Y5XRPAZTKAA, seg_pos={0 0}, upload=55892]: Uploading piece... Jul 26 21:50:02.084492603 (58h14m11.807055643s) [noise=true , address=79.127.205.245:4228 , piece_id=PJCWRMEA463F4KQEHDPSTJASJDJSUCFYTTJNKVVHA7XSO7A3MYLA, seg_pos={0 0}, upload=55892]: Uploading piece... Jul 26 21:50:02.085463742 (58h14m11.808026782s) [noise=true , address=79.127.219.38:4017 , piece_id=2W5HW5PQG25YM46GFQALDFSNDTG5MRJSU7L4LLRTGFQ3SQHRYKBA, seg_pos={0 0}, upload=55892]: Uploading piece... Jul 26 21:50:02.086414180 (58h14m11.80897722s) [noise=true , address=104.36.82.56:4248 , piece_id=OPCCILDHBTECTWDT7Q2JVPVWTTOVRROGILW4BC6UOOOTPTTYWL4Q, seg_pos={0 0}, upload=55892]: Uploading piece... Jul 26 21:50:02.086966296 (58h14m11.809529336s) [noise=true , address=104.36.82.54:4182 , piece_id=I6Y7XW37FUIC6IOCL24VR5FXIBCKK3C7Q3CJ35DQBAO66MCAYHQQ, seg_pos={0 0}, upload=55892]: Uploading piece... Jul 26 21:50:02.087477084 (58h14m11.810040134s) [noise=true , address=77.90.31.17:4780 , piece_id=VWGOT6WCBWGPRPIDICMDHD7PJ3HZMCLRB2DOIURHYA64HXGFII4A, seg_pos={0 0}, upload=55892]: Uploading piece... Jul 26 21:50:02.087769773 (58h14m11.810332823s) [noise=true , address=64.49.13.140:4510 , piece_id=OIPR65WGIT62BZTNIHORTUEQ5VETFWO3MZNOLCV5AVH3UU3KMXDQ, seg_pos={0 0}, upload=55892]: Uploading piece... Jul 26 21:50:02.088057153 (58h14m11.810620194s) [noise=true , address=104.36.82.13:4208 , piece_id=M526JFD6PEOJTPR5OAKZVAEBNRQZAOVQB7HSXRV67U5X46XYPAQQ, seg_pos={0 0}, upload=55892]: Uploading piece... Jul 26 21:50:02.088541139 (58h14m11.811104179s) [noise=true , address=104.36.82.55:4211 , piece_id=TL2UJO6QLBUQPQ2KY4PZEVGDSAYKMEY6UFAGUKUKLS2NHSUSTDEA, seg_pos={0 0}, upload=55892]: Uploading piece... Jul 26 21:50:02.088845693 (58h14m11.811408733s) [noise=true , address=104.36.82.53:4143 , piece_id=IVMX4BLEEWOOS5KOCZBLWOAEN6JSIHWG7CUAP6AJSQ7J3NQU54JA, seg_pos={0 0}, upload=55892]: Uploading piece... Jul 26 21:50:03.212728453 (58h14m12.935291493s) [upload=55892]: Next returned nil segment Jul 26 21:50:03.212747980 (58h14m12.935311021s) [upload=55892]: Waiting for error group managing segments... Jul 26 21:50:03.551213199 (58h14m13.273776239s) [noise=true , address=79.127.205.245:4228 , piece_id=PJCWRMEA463F4KQEHDPSTJASJDJSUCFYTTJNKVVHA7XSO7A3MYLA, seg_pos={0 0}, upload=55892]: Done uploading piece. err: <nil> Jul 26 21:50:03.677763357 (58h14m13.400326397s) [noise=true , address=64.49.13.140:4510 , piece_id=OIPR65WGIT62BZTNIHORTUEQ5VETFWO3MZNOLCV5AVH3UU3KMXDQ, seg_pos={0 0}, upload=55892]: Done uploading piece. err: <nil> Jul 26 21:50:03.990632711 (58h14m13.713195751s) [noise=true , address=79.127.219.38:4017 , piece_id=2W5HW5PQG25YM46GFQALDFSNDTG5MRJSU7L4LLRTGFQ3SQHRYKBA, seg_pos={0 0}, upload=55892]: Done uploading piece. err: <nil> Jul 26 21:50:04.261787182 (58h14m13.984350222s) [noise=true , address=79.127.205.234:4612 , piece_id=7HKF7KKVYTXSKWWUVU5KKYWJ6KTZNG4XJZZL32GB33PO5LEFS7DQ, seg_pos={0 0}, upload=55892]: Done uploading piece. err: <nil> Jul 26 21:50:32.945335427 (58h14m42.667898467s) [noise=true , address=77.90.31.17:4780 , piece_id=VWGOT6WCBWGPRPIDICMDHD7PJ3HZMCLRB2DOIURHYA64HXGFII4A, seg_pos={0 0}, upload=55892]: Done uploading piece. err: <nil> Jul 26 21:51:00.988546736 (58h15m10.711109776s) [noise=true , address=104.36.82.54:4182 , piece_id=I6Y7XW37FUIC6IOCL24VR5FXIBCKK3C7Q3CJ35DQBAO66MCAYHQQ, seg_pos={0 0}, upload=55892]: Done uploading piece. err: ecclient: upload failed (node:12HbkH31sfDUnZLXAbmzQphcFYmzB8MQo98Q6Ufbo59et42zxMy, address:104.36.82.54:4182): protocol: write tcp 188.166.225.170:45504->104.36.82.54:4182: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:51:00.988609455 (58h15m10.711172495s) [noise=true , address=104.36.82.47:4548 , piece_id=B64Y2AHXKFYVVTXLGSCICN6J66NPWR5BRXKCYGRCSB74HUCZ53RQ, seg_pos={0 0}, upload=55892]: Done uploading piece. err: ecclient: upload failed (node:1P7nZqYyfKc2CT4Sf8bWjEM6zk34vVz9UMifuKRybUjWKQqekT, address:104.36.82.47:4548): protocol: write tcp 188.166.225.170:34726->104.36.82.47:4548: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:51:00.988641557 (58h15m10.711204587s) [noise=true , address=104.36.82.13:4208 , piece_id=M526JFD6PEOJTPR5OAKZVAEBNRQZAOVQB7HSXRV67U5X46XYPAQQ, seg_pos={0 0}, upload=55892]: Done uploading piece. err: ecclient: upload failed (node:12SXSN3HpeYwgHRpFFST5wwP3hhXEe3QB8VzWTNNE19sFoWW4o5, address:104.36.82.13:4208): protocol: write tcp 188.166.225.170:56986->104.36.82.13:4208: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:51:00.988656986 (58h15m10.711220027s) [noise=true , address=104.36.82.55:4211 , piece_id=TL2UJO6QLBUQPQ2KY4PZEVGDSAYKMEY6UFAGUKUKLS2NHSUSTDEA, seg_pos={0 0}, upload=55892]: Done uploading piece. err: ecclient: upload failed (node:1269tDiDZTJSwmka4HLKMEaHErdCaUqXEYTFimGWhtjai9o4dAB, address:104.36.82.55:4211): protocol: write tcp 188.166.225.170:44704->104.36.82.55:4211: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:51:00.988678598 (58h15m10.711241638s) [noise=true , address=104.36.82.12:4171 , piece_id=7LREHRX6RZ3QQA4K2DWWL2J7ICSA67H5X36EOQFGFQBAWPEC2NEQ, seg_pos={0 0}, upload=55892]: Done uploading piece. err: ecclient: upload failed (node:1z2Fh5fPFTb9VZX6to5hdUYUGst63Ti1BVtdVtYk7vkrUqXMKM, address:104.36.82.12:4171): protocol: write tcp 188.166.225.170:42298->104.36.82.12:4171: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:51:00.988690410 (58h15m10.71125345s) [noise=true , address=104.36.82.56:4248 , piece_id=OPCCILDHBTECTWDT7Q2JVPVWTTOVRROGILW4BC6UOOOTPTTYWL4Q, seg_pos={0 0}, upload=55892]: Done uploading piece. err: ecclient: upload failed (node:1QpYBQ2PfiAwvt5EmFvjBnKgBhwnuyF6b77qJocpNe1YqSzP77, address:104.36.82.56:4248): protocol: rpc: dial tcp 104.36.82.56:4248: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:51:00.988710339 (58h15m10.711273379s) [noise=true , address=104.36.82.7:4919 , piece_id=Q3NJG75JW7JDWYJ76OGVL4L2Y2YVDBCQGBAUUDV2DCYPBOAGWG4Q, seg_pos={0 0}, upload=55892]: Done uploading piece. err: ecclient: upload failed (node:182cMUjmC3Ku6z4KucQpqTS8NcKz7VJKeXt9tFYPnUtDUj6A2w, address:104.36.82.7:4919): protocol: write tcp 188.166.225.170:49696->104.36.82.7:4919: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:51:00.988732521 (58h15m10.711295561s) [noise=true , address=104.36.82.53:4143 , piece_id=IVMX4BLEEWOOS5KOCZBLWOAEN6JSIHWG7CUAP6AJSQ7J3NQU54JA, seg_pos={0 0}, upload=55892]: Done uploading piece. err: ecclient: upload failed (node:12EBnXkxXyf7TYw89gDnoHNpKPyBpNhdT46CipJwjqr6Cg6jxMQ, address:104.36.82.53:4143): protocol: write tcp 188.166.225.170:59516->104.36.82.53:4143: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:51:00.988738653 (58h15m10.711301693s) [noise=true , address=104.36.82.49:4980 , piece_id=MGHOGY5YX3COWNM5SPZPAULE4AJROYPMPKIZ6N4EM5DYNR2C2FNQ, seg_pos={0 0}, upload=55892]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:104.36.82.49:4980): protocol: write tcp 188.166.225.170:46268->104.36.82.49:4980: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:51:00.988744123 (58h15m10.711307163s) [noise=true , address=104.36.82.41:4761 , piece_id=ARSSUFV5APYLOM2OIGEKJWOC2ZGP3KLINKUMORYWNX4KNQRFMWLQ, seg_pos={0 0}, upload=55892]: Done uploading piece. err: ecclient: upload failed (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE, address:104.36.82.41:4761): protocol: write tcp 188.166.225.170:38682->104.36.82.41:4761: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:51:00.988764272 (58h15m10.711327312s) [noise=true , address=104.36.82.12:4169 , piece_id=XAHRR6PMD5EFWADGHASVA66R27PFCYR5A6XC7QON7Y5XRPAZTKAA, seg_pos={0 0}, upload=55892]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:104.36.82.12:4169): protocol: write tcp 188.166.225.170:59222->104.36.82.12:4169: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:51:00.988779931 (58h15m10.711342971s) [noise=true , address=104.36.82.46:4496 , piece_id=ENDTXOCBDUTZVNODP6LR6SDCETRRPGEHTRBM4LO3RXD7U6EVPT5Q, seg_pos={0 0}, upload=55892]: Done uploading piece. err: ecclient: upload failed (node:12V8XhSWkyGUVYFR8MykifFAP1YAfGCwzqucDcMMfzw3xsVW77Z, address:104.36.82.46:4496): protocol: write tcp 188.166.225.170:45790->104.36.82.46:4496: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:51:00.988806973 (58h15m10.711370003s) [noise=true , address=104.36.82.15:4263 , piece_id=IC7GTUCC3JUECQYYKSEUZENP2IBE3YM3ACP5746AM5PNPSCX5DYA, seg_pos={0 0}, upload=55892]: Done uploading piece. err: ecclient: upload failed (node:1f97yBEvcrYHqMq7iovVoju77jzhUYgoQCsUGULeTagugFXrCu, address:104.36.82.15:4263): protocol: write tcp 188.166.225.170:34274->104.36.82.15:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:51:00.988826781 (58h15m10.711389821s) [seg_pos={0 0}, upload=55892]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 26 21:51:01.220364944 (58h15m10.942927984s) [upload=55892]: Done uploading segments... Jul 26 21:51:01.220392858 (58h15m10.942955898s) [upload=55892]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 1m0.233661879s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [................................X....X......XX....XX.XX.XXX.] duration: 23.830071837s FAIL (2025-07-26 21:50:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== :20.720025068 (58h13m30.442588108s) [upload=55880]: Uploading segments... Jul 26 21:49:20.720154165 (58h13m30.442717205s) [upload=55880]: Got next segment. Inline: false Jul 26 21:49:20.720188190 (58h13m30.44275124s) [upload=55880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:49:20.720195223 (58h13m30.442758263s) [upload=55880]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:49:20.954801116 (58h13m30.677364156s) [seg_pos={0 0}, upload=55880]: Begin upload segment... Jul 26 21:49:20.954848577 (58h13m30.677411617s) [seg_pos={0 0}, upload=55880]: Done begin upload segment. Jul 26 21:49:20.954871531 (58h13m30.677434571s) [noise=true , address=104.36.82.10:4079 , piece_id=6ZY2KH5GRH66PUXAXD6KLFNOOYTBAZGMRC65OTN2OYLXI2TU7PVQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.954909283 (58h13m30.677472323s) [noise=true , address=45.91.116.11:4118 , piece_id=D7PNJMUFVKPAASPLYRBLWK7NOLLXPQKML6BRSGXJSRN2AQDSQ7OQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.955274702 (58h13m30.677837742s) [noise=true , address=45.91.116.12:4227 , piece_id=HTPKZWEVA6PYSPXK6GU4C5BR2Z3PWPNYEC76YG6J7CH5FUWH4TLA, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.955577211 (58h13m30.678140261s) [noise=true , address=45.91.116.11:4117 , piece_id=W4ULMCMA4V2IUD4LH5E5RLF42II4GTAQT27KH7LHMLZTTIZMVMDQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.955876093 (58h13m30.678439133s) [noise=true , address=45.91.116.9:4262 , piece_id=XIDUV5PBNP25NQSDNOXTDBXYZEBQ7WCKMIWQBJCI3TRNLGEJXS6A, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.955880962 (58h13m30.678444002s) [noise=true , address=104.36.82.15:4258 , piece_id=X5GYDLYO52PMICN2HAARX5YOKWEL4NUCTB4LOLXPMO3EKZF3KRLA, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.956187629 (58h13m30.678750669s) [noise=true , address=45.91.116.5:4299 , piece_id=WEWBO4XKYKBXAYORPQ5ZRBUUVSLUIU4CGT5JQYQSD4PHAR522ICA, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.956559070 (58h13m30.67912211s) [noise=true , address=45.91.116.14:4190 , piece_id=NHMEFPDYVBJTKYW3IU2E4JT7ZHIXT3OMR2YMT6XZU6Z6D656BJRQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.956847902 (58h13m30.679410932s) [noise=true , address=77.90.31.13:4626 , piece_id=6IWCX5BBRTOXBG6RIZTGT2THHJOAFO5HISUGD3ZL2WRABPDFSCJA, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.957083984 (58h13m30.679647024s) [noise=true , address=45.91.116.5:4297 , piece_id=3IT24CPL2UBY3V5VFJBBQCFTHVPRTD6LPCAVSWVPG5MNX23N2HVQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.957305738 (58h13m30.679868778s) [noise=true , address=45.91.116.6:4974 , piece_id=TRJODTTAUELXNW67TLIVKQPIFTQL4B3U3XZJIC2WZLWEZEKLHCYQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.957589441 (58h13m30.680152481s) [noise=true , address=104.36.82.13:4186 , piece_id=5W527C73JPQ235TJQUZDDNWLIAZNMEDSZH3ON3F6F6ODINYPEUDQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.957858666 (58h13m30.680421706s) [noise=true , address=77.90.31.10:4518 , piece_id=E4JEKT3K6OIV5HC26C4FKU7A2RSRSDEZAHOBH2CHE6YZYHBO3PRQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.958147608 (58h13m30.680710649s) [noise=true , address=104.36.82.7:4911 , piece_id=WF5UOSGZEMFZ3PUPJE4XJ2WNKSXCMQWQMENFRJ3UWKKNLQAXCOUQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.958371518 (58h13m30.680934558s) [noise=true , address=45.91.116.7:4011 , piece_id=D6T4PW3PUWEWIGXAUYWNXUAUNFEPNQHXIHPORLBV3R37PSOV2R7Q, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.958530361 (58h13m30.681093401s) [noise=true , address=104.36.82.4:4803 , piece_id=XZORP2CICQ6BLEDCQG7T5KDXZ4LDNAAFHLULCL3AKA3CTQEKEN5Q, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:22.690435183 (58h13m32.412998223s) [upload=55880]: Next returned nil segment Jul 26 21:49:22.690453668 (58h13m32.413016698s) [upload=55880]: Waiting for error group managing segments... Jul 26 21:49:22.966260902 (58h13m32.688823942s) [noise=true , address=45.91.116.12:4227 , piece_id=HTPKZWEVA6PYSPXK6GU4C5BR2Z3PWPNYEC76YG6J7CH5FUWH4TLA, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:22.972084183 (58h13m32.694647223s) [noise=true , address=45.91.116.5:4299 , piece_id=WEWBO4XKYKBXAYORPQ5ZRBUUVSLUIU4CGT5JQYQSD4PHAR522ICA, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.066449243 (58h13m32.789012273s) [noise=true , address=45.91.116.14:4190 , piece_id=NHMEFPDYVBJTKYW3IU2E4JT7ZHIXT3OMR2YMT6XZU6Z6D656BJRQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.074262652 (58h13m32.796825692s) [noise=true , address=45.91.116.11:4117 , piece_id=W4ULMCMA4V2IUD4LH5E5RLF42II4GTAQT27KH7LHMLZTTIZMVMDQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.112159149 (58h13m32.834722189s) [noise=true , address=45.91.116.9:4262 , piece_id=XIDUV5PBNP25NQSDNOXTDBXYZEBQ7WCKMIWQBJCI3TRNLGEJXS6A, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.124382326 (58h13m32.846945366s) [noise=true , address=45.91.116.6:4974 , piece_id=TRJODTTAUELXNW67TLIVKQPIFTQL4B3U3XZJIC2WZLWEZEKLHCYQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.137504975 (58h13m32.860068015s) [noise=true , address=45.91.116.7:4011 , piece_id=D6T4PW3PUWEWIGXAUYWNXUAUNFEPNQHXIHPORLBV3R37PSOV2R7Q, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.143214057 (58h13m32.865777157s) [noise=true , address=45.91.116.11:4118 , piece_id=D7PNJMUFVKPAASPLYRBLWK7NOLLXPQKML6BRSGXJSRN2AQDSQ7OQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.205203492 (58h13m32.927766532s) [noise=true , address=45.91.116.5:4297 , piece_id=3IT24CPL2UBY3V5VFJBBQCFTHVPRTD6LPCAVSWVPG5MNX23N2HVQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:50.072705664 (58h13m59.795268704s) [noise=true , address=104.36.82.13:4186 , piece_id=5W527C73JPQ235TJQUZDDNWLIAZNMEDSZH3ON3F6F6ODINYPEUDQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:45962->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:50.072761511 (58h13m59.795324541s) [noise=true , address=104.36.82.10:4079 , piece_id=6ZY2KH5GRH66PUXAXD6KLFNOOYTBAZGMRC65OTN2OYLXI2TU7PVQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:50.072824261 (58h13m59.795387351s) [noise=true , address=104.36.82.4:4803 , piece_id=XZORP2CICQ6BLEDCQG7T5KDXZ4LDNAAFHLULCL3AKA3CTQEKEN5Q, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 188.166.225.170:58540->104.36.82.4:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:50.073041026 (58h13m59.795604056s) [noise=true , address=104.36.82.15:4258 , piece_id=X5GYDLYO52PMICN2HAARX5YOKWEL4NUCTB4LOLXPMO3EKZF3KRLA, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:50.073108805 (58h13m59.795671845s) [noise=true , address=77.90.31.13:4626 , piece_id=6IWCX5BBRTOXBG6RIZTGT2THHJOAFO5HISUGD3ZL2WRABPDFSCJA, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:44868->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:44868->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44868->77.90.31.13:4626: use of closed network connection Jul 26 21:49:50.073139484 (58h13m59.795702524s) [noise=true , address=77.90.31.10:4518 , piece_id=E4JEKT3K6OIV5HC26C4FKU7A2RSRSDEZAHOBH2CHE6YZYHBO3PRQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55880->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55880->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55880->77.90.31.10:4518: use of closed network connection Jul 26 21:49:50.073184069 (58h13m59.795747109s) [noise=true , address=104.36.82.7:4911 , piece_id=WF5UOSGZEMFZ3PUPJE4XJ2WNKSXCMQWQMENFRJ3UWKKNLQAXCOUQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:54892->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:50.073219787 (58h13m59.795782837s) [seg_pos={0 0}, upload=55880]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 26 21:49:50.320347248 (58h14m0.042910298s) [upload=55880]: Done uploading segments... Jul 26 21:49:50.320374751 (58h14m0.042937841s) [upload=55880]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 23.830102126s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [........X.........X.................X.......................] duration: 1.880351287s FAIL (2025-07-26 21:50:47 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.880384279s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 11 out of 60 is failed (threshold=3) [...X...................................XX.........XX.XX.XXXX] duration: 30.235601917s FAIL (2025-07-26 21:50:26 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.235639429s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........X.........X.................X......................] duration: 2.041695328s FAIL (2025-07-26 21:50:25 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 2.041729382s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 21 out of 60 is failed (threshold=3) [.......................................XXXXXXXXXXXXXXXXXXXXX] duration: 1m0.261839265s FAIL (2025-07-26 21:50:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== ece_id=JY4A3SR2SYGQX4JBN7PU7CNEM4SV6YYNV2QL5Q36HOZ3YVTOVNUQ, seg_pos={0 0}, upload=55877]: Uploading piece... Jul 26 21:49:01.581092167 (58h13m11.303655207s) [noise=true , address=104.36.82.4:4815 , piece_id=KKHOPYRUM43RMD6RQUPZRBPRBK5SFGPZFWVWNUZO64KI37B37USQ, seg_pos={0 0}, upload=55877]: Uploading piece... Jul 26 21:49:01.581263916 (58h13m11.303826956s) [noise=true , address=104.36.82.56:4263 , piece_id=3AL4SR3GYN2T3G2DOGZSSJJQI5EIRG56CFRMW5ZJC2K2KOLEJYBQ, seg_pos={0 0}, upload=55877]: Uploading piece... Jul 26 21:49:01.581342647 (58h13m11.303905687s) [noise=true , address=79.127.205.246:4280 , piece_id=NPTBSNBRSZ4NC5WARPHO5FS76SXT7KU5WPC3J462LIIN4AM4BKNA, seg_pos={0 0}, upload=55877]: Uploading piece... Jul 26 21:49:01.581484117 (58h13m11.304047157s) [noise=true , address=104.36.82.52:4090 , piece_id=R56QDHHK5CIBZSAY3UA62MU6WYJEQV4CUI2ZXY3LJT5XJZ7ILSEQ, seg_pos={0 0}, upload=55877]: Uploading piece... Jul 26 21:49:01.581558079 (58h13m11.304121119s) [noise=true , address=104.36.82.17:4354 , piece_id=OMOLQZAZUBDJM4IQEEHEA7MHMSHG4CQYSJXX2O2NWCRWRFIAAFDQ, seg_pos={0 0}, upload=55877]: Uploading piece... Jul 26 21:49:01.581693458 (58h13m11.304256498s) [noise=true , address=104.36.82.44:4885 , piece_id=6TAQQ7ZE2BZ6JA44GCME7SJXRITDEEJNNEWZ44O7TFUDTAZ2J73A, seg_pos={0 0}, upload=55877]: Uploading piece... Jul 26 21:49:01.581765957 (58h13m11.304328997s) [noise=true , address=104.36.82.47:4549 , piece_id=Y6LRTWIZJDZV7AO6CLQJL7B4TU4FM2ICYGEGAAF3SLH7MQG7PYCQ, seg_pos={0 0}, upload=55877]: Uploading piece... Jul 26 21:49:01.581977842 (58h13m11.304540882s) [noise=true , address=77.90.31.15:4726 , piece_id=DK74F2FKNBFFKGWMDFFOURHU3H4AVY46FSOYBEB4AAOXOD3QW2BQ, seg_pos={0 0}, upload=55877]: Uploading piece... Jul 26 21:49:01.582005715 (58h13m11.304568755s) [noise=true , address=104.36.82.46:4501 , piece_id=4XV4VQYSAXWPGYLSRV3N6SO642O62NN5IDGCC55GXYINEBED4LPA, seg_pos={0 0}, upload=55877]: Uploading piece... Jul 26 21:49:01.582210728 (58h13m11.304773768s) [noise=true , address=104.36.82.14:4248 , piece_id=54U6NTII4GSWIBYI3DK2PZXCT4CB473CTUCUSLIWWQUIR5FL36FA, seg_pos={0 0}, upload=55877]: Uploading piece... Jul 26 21:49:01.582438213 (58h13m11.305001253s) [noise=true , address=45.91.116.9:4269 , piece_id=WOPMGXD5E6MY4CM3QJ4TFGL5FASKF3HEPC66QWBMLC2AIFRTGNRA, seg_pos={0 0}, upload=55877]: Uploading piece... Jul 26 21:49:02.682949085 (58h13m12.405512125s) [upload=55877]: Next returned nil segment Jul 26 21:49:02.682992419 (58h13m12.405555459s) [upload=55877]: Waiting for error group managing segments... Jul 26 21:49:02.972302718 (58h13m12.694865758s) [noise=true , address=79.127.205.246:4280 , piece_id=NPTBSNBRSZ4NC5WARPHO5FS76SXT7KU5WPC3J462LIIN4AM4BKNA, seg_pos={0 0}, upload=55877]: Done uploading piece. err: <nil> Jul 26 21:49:03.049712928 (58h13m12.772275968s) [noise=true , address=79.127.205.235:4685 , piece_id=5RHRJMP2JWC4FOVJ3AMTBXWBHJBKIJEV52MXSEROLJVQFMHKTMHQ, seg_pos={0 0}, upload=55877]: Done uploading piece. err: <nil> Jul 26 21:49:03.552081732 (58h13m13.274644773s) [noise=true , address=45.91.116.9:4269 , piece_id=WOPMGXD5E6MY4CM3QJ4TFGL5FASKF3HEPC66QWBMLC2AIFRTGNRA, seg_pos={0 0}, upload=55877]: Done uploading piece. err: <nil> Jul 26 21:49:03.604168600 (58h13m13.32673165s) [noise=true , address=79.127.219.38:4003 , piece_id=JY4A3SR2SYGQX4JBN7PU7CNEM4SV6YYNV2QL5Q36HOZ3YVTOVNUQ, seg_pos={0 0}, upload=55877]: Done uploading piece. err: <nil> Jul 26 21:49:05.795316026 (58h13m15.517879066s) [noise=true , address=64.49.13.138:4482 , piece_id=XMUYNG6INO52RR2Y6KVFRBA6EBGWSTPOOIH7TWBP3FTD2AGHSXFA, seg_pos={0 0}, upload=55877]: Done uploading piece. err: <nil> Jul 26 21:49:16.992131289 (58h13m26.71469432s) [noise=true , address=77.90.31.15:4726 , piece_id=DK74F2FKNBFFKGWMDFFOURHU3H4AVY46FSOYBEB4AAOXOD3QW2BQ, seg_pos={0 0}, upload=55877]: Done uploading piece. err: <nil> Jul 26 21:50:00.486437776 (58h14m10.209000816s) [noise=true , address=104.36.82.44:4885 , piece_id=6TAQQ7ZE2BZ6JA44GCME7SJXRITDEEJNNEWZ44O7TFUDTAZ2J73A, seg_pos={0 0}, upload=55877]: Done uploading piece. err: ecclient: upload failed (node:118qGQLFSNbsYsvFrKjTXXWbFYu2sRaRHrUNkKNCrNVpzVQRtG, address:104.36.82.44:4885): protocol: write tcp 188.166.225.170:48114->104.36.82.44:4885: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:50:00.486488814 (58h14m10.209051854s) [noise=true , address=104.36.82.52:4090 , piece_id=R56QDHHK5CIBZSAY3UA62MU6WYJEQV4CUI2ZXY3LJT5XJZ7ILSEQ, seg_pos={0 0}, upload=55877]: Done uploading piece. err: ecclient: upload failed (node:12UUnoD9p5FC7rq9fMSzu1phkFpsnPsu4UYxC59MhpMjJ6wUC6F, address:104.36.82.52:4090): protocol: rpc: dial tcp 104.36.82.52:4090: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:50:00.486522508 (58h14m10.209085558s) [noise=true , address=104.36.82.17:4354 , piece_id=OMOLQZAZUBDJM4IQEEHEA7MHMSHG4CQYSJXX2O2NWCRWRFIAAFDQ, seg_pos={0 0}, upload=55877]: Done uploading piece. err: ecclient: upload failed (node:16LBUBhBQLy2yyVYjwhFtoahB9nE2KDyHvNWbdw7zXC1RgHjdh, address:104.36.82.17:4354): protocol: write tcp 188.166.225.170:50410->104.36.82.17:4354: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:50:00.486559129 (58h14m10.209122169s) [noise=true , address=104.36.82.53:4131 , piece_id=OYU5BSPZUEIVZTQP6XPHM3WLISW5UJ736WA2Y3G3BHI36NMAVHNA, seg_pos={0 0}, upload=55877]: Done uploading piece. err: ecclient: upload failed (node:1uhyc5ozrF9ndeEnQXtjc9o3tsYiAnW1dzeuHV3kv1JS4T4a8C, address:104.36.82.53:4131): protocol: write tcp 188.166.225.170:33640->104.36.82.53:4131: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:50:00.486589256 (58h14m10.209152286s) [noise=true , address=104.36.82.14:4248 , piece_id=54U6NTII4GSWIBYI3DK2PZXCT4CB473CTUCUSLIWWQUIR5FL36FA, seg_pos={0 0}, upload=55877]: Done uploading piece. err: ecclient: upload failed (node:1u6ir2PLqDa9xGm64cuqq1RqY4qeBa1fqCo9VghspBf54XEXqn, address:104.36.82.14:4248): protocol: write tcp 188.166.225.170:50758->104.36.82.14:4248: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:50:00.486630815 (58h14m10.209193845s) [noise=true , address=104.36.82.6:4899 , piece_id=3WU75ZCRKKVEBYLLJQWU2CZGDKCRJ5USJCD72VXSCWSYJCQGIEWQ, seg_pos={0 0}, upload=55877]: Done uploading piece. err: ecclient: upload failed (node:1tdNNPMJKCqJ39U1f4aj3Xrwh7J12jpyQGoQDQUdWNCxszcA5k, address:104.36.82.6:4899): protocol: write tcp 188.166.225.170:35982->104.36.82.6:4899: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:50:00.486669861 (58h14m10.209232891s) [noise=true , address=104.36.82.54:4184 , piece_id=DM2WCVBYV4CSOYSNCEBEL74ACMDJY56S2VDHETZ6KMAKSVJELCXQ, seg_pos={0 0}, upload=55877]: Done uploading piece. err: ecclient: upload failed (node:1VYcRgiFADDQ1rLsiMufThN1K51cuoqKRXNjbp2w9Exu1d3gQG, address:104.36.82.54:4184): protocol: write tcp 188.166.225.170:56516->104.36.82.54:4184: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:50:00.486710197 (58h14m10.209273237s) [noise=true , address=104.36.82.46:4501 , piece_id=4XV4VQYSAXWPGYLSRV3N6SO642O62NN5IDGCC55GXYINEBED4LPA, seg_pos={0 0}, upload=55877]: Done uploading piece. err: ecclient: upload failed (node:1hmsSAiSp1CBFqKhVE2S3sFfqYYkGVAiV1GfrT4Rd7hBaFNPpT, address:104.36.82.46:4501): protocol: write tcp 188.166.225.170:37132->104.36.82.46:4501: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:50:00.486720648 (58h14m10.209283678s) [noise=true , address=104.36.82.4:4815 , piece_id=KKHOPYRUM43RMD6RQUPZRBPRBK5SFGPZFWVWNUZO64KI37B37USQ, seg_pos={0 0}, upload=55877]: Done uploading piece. err: ecclient: upload failed (node:1D4gdLHtXGAiLFhppWTVvqXjmiU6r5KJPyW1HG8zzyZ3EC6eYa, address:104.36.82.4:4815): protocol: write tcp 188.166.225.170:35104->104.36.82.4:4815: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:50:00.486758229 (58h14m10.209321259s) [noise=true , address=104.36.82.56:4263 , piece_id=3AL4SR3GYN2T3G2DOGZSSJJQI5EIRG56CFRMW5ZJC2K2KOLEJYBQ, seg_pos={0 0}, upload=55877]: Done uploading piece. err: ecclient: upload failed (node:12tLW4r8AQKLCbSWZoNUVi5v2YtF8NthUmX81TLQn8G8TotuByG, address:104.36.82.56:4263): protocol: write tcp 188.166.225.170:40230->104.36.82.56:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:50:00.486764541 (58h14m10.209327581s) [noise=true , address=104.36.82.47:4549 , piece_id=Y6LRTWIZJDZV7AO6CLQJL7B4TU4FM2ICYGEGAAF3SLH7MQG7PYCQ, seg_pos={0 0}, upload=55877]: Done uploading piece. err: ecclient: upload failed (node:1HWoyMKQ6v2XWUfEmfFh3m8AiGivAcYkkZ9na8kqHFzH2hPDeK, address:104.36.82.47:4549): protocol: write tcp 188.166.225.170:54040->104.36.82.47:4549: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:50:00.486782546 (58h14m10.209345576s) [noise=true , address=104.36.82.42:4824 , piece_id=27ZEZMCB3JN7R5LHTNN36U575QQWEQRJ4VBU7W2KWOOPFSQXHRZA, seg_pos={0 0}, upload=55877]: Done uploading piece. err: ecclient: upload failed (node:12QTjnomQYFETHbaJBWxWEPo5yweArk68bKPeooZXpRZrpw1ACq, address:104.36.82.42:4824): protocol: write tcp 188.166.225.170:46842->104.36.82.42:4824: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:50:00.486810520 (58h14m10.20937356s) [seg_pos={0 0}, upload=55877]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Jul 26 21:50:00.745890508 (58h14m10.468453588s) [upload=55877]: Done uploading segments... Jul 26 21:50:00.745920786 (58h14m10.468483856s) [upload=55877]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 1m0.261855525s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..........X.........X.................X.....................] duration: 2.293798507s FAIL (2025-07-26 21:49:59 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 2.293818495s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.................................X....X......XX....XX.XX.XXX] duration: 30.249472132s FAIL (2025-07-26 21:49:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== :20.720025068 (58h13m30.442588108s) [upload=55880]: Uploading segments... Jul 26 21:49:20.720154165 (58h13m30.442717205s) [upload=55880]: Got next segment. Inline: false Jul 26 21:49:20.720188190 (58h13m30.44275124s) [upload=55880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:49:20.720195223 (58h13m30.442758263s) [upload=55880]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:49:20.954801116 (58h13m30.677364156s) [seg_pos={0 0}, upload=55880]: Begin upload segment... Jul 26 21:49:20.954848577 (58h13m30.677411617s) [seg_pos={0 0}, upload=55880]: Done begin upload segment. Jul 26 21:49:20.954871531 (58h13m30.677434571s) [noise=true , address=104.36.82.10:4079 , piece_id=6ZY2KH5GRH66PUXAXD6KLFNOOYTBAZGMRC65OTN2OYLXI2TU7PVQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.954909283 (58h13m30.677472323s) [noise=true , address=45.91.116.11:4118 , piece_id=D7PNJMUFVKPAASPLYRBLWK7NOLLXPQKML6BRSGXJSRN2AQDSQ7OQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.955274702 (58h13m30.677837742s) [noise=true , address=45.91.116.12:4227 , piece_id=HTPKZWEVA6PYSPXK6GU4C5BR2Z3PWPNYEC76YG6J7CH5FUWH4TLA, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.955577211 (58h13m30.678140261s) [noise=true , address=45.91.116.11:4117 , piece_id=W4ULMCMA4V2IUD4LH5E5RLF42II4GTAQT27KH7LHMLZTTIZMVMDQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.955876093 (58h13m30.678439133s) [noise=true , address=45.91.116.9:4262 , piece_id=XIDUV5PBNP25NQSDNOXTDBXYZEBQ7WCKMIWQBJCI3TRNLGEJXS6A, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.955880962 (58h13m30.678444002s) [noise=true , address=104.36.82.15:4258 , piece_id=X5GYDLYO52PMICN2HAARX5YOKWEL4NUCTB4LOLXPMO3EKZF3KRLA, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.956187629 (58h13m30.678750669s) [noise=true , address=45.91.116.5:4299 , piece_id=WEWBO4XKYKBXAYORPQ5ZRBUUVSLUIU4CGT5JQYQSD4PHAR522ICA, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.956559070 (58h13m30.67912211s) [noise=true , address=45.91.116.14:4190 , piece_id=NHMEFPDYVBJTKYW3IU2E4JT7ZHIXT3OMR2YMT6XZU6Z6D656BJRQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.956847902 (58h13m30.679410932s) [noise=true , address=77.90.31.13:4626 , piece_id=6IWCX5BBRTOXBG6RIZTGT2THHJOAFO5HISUGD3ZL2WRABPDFSCJA, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.957083984 (58h13m30.679647024s) [noise=true , address=45.91.116.5:4297 , piece_id=3IT24CPL2UBY3V5VFJBBQCFTHVPRTD6LPCAVSWVPG5MNX23N2HVQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.957305738 (58h13m30.679868778s) [noise=true , address=45.91.116.6:4974 , piece_id=TRJODTTAUELXNW67TLIVKQPIFTQL4B3U3XZJIC2WZLWEZEKLHCYQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.957589441 (58h13m30.680152481s) [noise=true , address=104.36.82.13:4186 , piece_id=5W527C73JPQ235TJQUZDDNWLIAZNMEDSZH3ON3F6F6ODINYPEUDQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.957858666 (58h13m30.680421706s) [noise=true , address=77.90.31.10:4518 , piece_id=E4JEKT3K6OIV5HC26C4FKU7A2RSRSDEZAHOBH2CHE6YZYHBO3PRQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.958147608 (58h13m30.680710649s) [noise=true , address=104.36.82.7:4911 , piece_id=WF5UOSGZEMFZ3PUPJE4XJ2WNKSXCMQWQMENFRJ3UWKKNLQAXCOUQ, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.958371518 (58h13m30.680934558s) [noise=true , address=45.91.116.7:4011 , piece_id=D6T4PW3PUWEWIGXAUYWNXUAUNFEPNQHXIHPORLBV3R37PSOV2R7Q, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:20.958530361 (58h13m30.681093401s) [noise=true , address=104.36.82.4:4803 , piece_id=XZORP2CICQ6BLEDCQG7T5KDXZ4LDNAAFHLULCL3AKA3CTQEKEN5Q, seg_pos={0 0}, upload=55880]: Uploading piece... Jul 26 21:49:22.690435183 (58h13m32.412998223s) [upload=55880]: Next returned nil segment Jul 26 21:49:22.690453668 (58h13m32.413016698s) [upload=55880]: Waiting for error group managing segments... Jul 26 21:49:22.966260902 (58h13m32.688823942s) [noise=true , address=45.91.116.12:4227 , piece_id=HTPKZWEVA6PYSPXK6GU4C5BR2Z3PWPNYEC76YG6J7CH5FUWH4TLA, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:22.972084183 (58h13m32.694647223s) [noise=true , address=45.91.116.5:4299 , piece_id=WEWBO4XKYKBXAYORPQ5ZRBUUVSLUIU4CGT5JQYQSD4PHAR522ICA, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.066449243 (58h13m32.789012273s) [noise=true , address=45.91.116.14:4190 , piece_id=NHMEFPDYVBJTKYW3IU2E4JT7ZHIXT3OMR2YMT6XZU6Z6D656BJRQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.074262652 (58h13m32.796825692s) [noise=true , address=45.91.116.11:4117 , piece_id=W4ULMCMA4V2IUD4LH5E5RLF42II4GTAQT27KH7LHMLZTTIZMVMDQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.112159149 (58h13m32.834722189s) [noise=true , address=45.91.116.9:4262 , piece_id=XIDUV5PBNP25NQSDNOXTDBXYZEBQ7WCKMIWQBJCI3TRNLGEJXS6A, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.124382326 (58h13m32.846945366s) [noise=true , address=45.91.116.6:4974 , piece_id=TRJODTTAUELXNW67TLIVKQPIFTQL4B3U3XZJIC2WZLWEZEKLHCYQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.137504975 (58h13m32.860068015s) [noise=true , address=45.91.116.7:4011 , piece_id=D6T4PW3PUWEWIGXAUYWNXUAUNFEPNQHXIHPORLBV3R37PSOV2R7Q, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.143214057 (58h13m32.865777157s) [noise=true , address=45.91.116.11:4118 , piece_id=D7PNJMUFVKPAASPLYRBLWK7NOLLXPQKML6BRSGXJSRN2AQDSQ7OQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:23.205203492 (58h13m32.927766532s) [noise=true , address=45.91.116.5:4297 , piece_id=3IT24CPL2UBY3V5VFJBBQCFTHVPRTD6LPCAVSWVPG5MNX23N2HVQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: <nil> Jul 26 21:49:50.072705664 (58h13m59.795268704s) [noise=true , address=104.36.82.13:4186 , piece_id=5W527C73JPQ235TJQUZDDNWLIAZNMEDSZH3ON3F6F6ODINYPEUDQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:104.36.82.13:4186): protocol: write tcp 188.166.225.170:45962->104.36.82.13:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:50.072761511 (58h13m59.795324541s) [noise=true , address=104.36.82.10:4079 , piece_id=6ZY2KH5GRH66PUXAXD6KLFNOOYTBAZGMRC65OTN2OYLXI2TU7PVQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:104.36.82.10:4079): protocol: rpc: dial tcp 104.36.82.10:4079: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:50.072824261 (58h13m59.795387351s) [noise=true , address=104.36.82.4:4803 , piece_id=XZORP2CICQ6BLEDCQG7T5KDXZ4LDNAAFHLULCL3AKA3CTQEKEN5Q, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 188.166.225.170:58540->104.36.82.4:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:50.073041026 (58h13m59.795604056s) [noise=true , address=104.36.82.15:4258 , piece_id=X5GYDLYO52PMICN2HAARX5YOKWEL4NUCTB4LOLXPMO3EKZF3KRLA, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:104.36.82.15:4258): protocol: rpc: dial tcp 104.36.82.15:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:50.073108805 (58h13m59.795671845s) [noise=true , address=77.90.31.13:4626 , piece_id=6IWCX5BBRTOXBG6RIZTGT2THHJOAFO5HISUGD3ZL2WRABPDFSCJA, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:44868->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:44868->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44868->77.90.31.13:4626: use of closed network connection Jul 26 21:49:50.073139484 (58h13m59.795702524s) [noise=true , address=77.90.31.10:4518 , piece_id=E4JEKT3K6OIV5HC26C4FKU7A2RSRSDEZAHOBH2CHE6YZYHBO3PRQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: write tcp 188.166.225.170:55880->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:55880->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55880->77.90.31.10:4518: use of closed network connection Jul 26 21:49:50.073184069 (58h13m59.795747109s) [noise=true , address=104.36.82.7:4911 , piece_id=WF5UOSGZEMFZ3PUPJE4XJ2WNKSXCMQWQMENFRJ3UWKKNLQAXCOUQ, seg_pos={0 0}, upload=55880]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:104.36.82.7:4911): protocol: write tcp 188.166.225.170:54892->104.36.82.7:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:50.073219787 (58h13m59.795782837s) [seg_pos={0 0}, upload=55880]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Jul 26 21:49:50.320347248 (58h14m0.042910298s) [upload=55880]: Done uploading segments... Jul 26 21:49:50.320374751 (58h14m0.042937841s) [upload=55880]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.249487713s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...........X.........X.................X....................] duration: 2.00411806s FAIL (2025-07-26 21:49:28 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 2.004158497s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [....X...................................XX.........XX.XX.XXX] duration: 30.273623342s FAIL (2025-07-26 21:49:26 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.273643951s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..................................X....X......XX....XX.XX.XX] duration: 30.230619106s FAIL (2025-07-26 21:49:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== nt: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:48:49.118680675 (58h12m58.841243715s) [seg_pos={0 0}, upload=55863]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 21:48:49.359136580 (58h12m59.08169966s) [upload=55863]: Done uploading segments... Jul 26 21:48:49.359164624 (58h12m59.081727694s) [upload=55863]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 26 21:48:50.250673444 (58h12m59.973236484s) [upload=55871]: Starting upload Jul 26 21:48:50.250734461 (58h12m59.973297501s) [upload=55871]: Done starting upload Jul 26 21:48:50.250829623 (58h12m59.973392663s) [upload=55871]: Uploading segments... Jul 26 21:48:50.250962978 (58h12m59.973526018s) [upload=55871]: Got next segment. Inline: false Jul 26 21:48:50.250993166 (58h12m59.973556196s) [upload=55871]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:48:50.250997694 (58h12m59.973560734s) [upload=55871]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:48:50.481839855 (58h13m0.204402935s) [seg_pos={0 0}, upload=55871]: Begin upload segment... Jul 26 21:48:50.481908065 (58h13m0.204471155s) [seg_pos={0 0}, upload=55871]: Done begin upload segment. Jul 26 21:48:50.481944054 (58h13m0.204507124s) [noise=true , address=45.91.116.7:4009 , piece_id=L5NGIGVAFIEAZWXJ62L6CSWVQUOALVLXNZQOERUY6TT5JHX5VDGQ, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.481950527 (58h13m0.204513577s) [noise=true , address=77.90.31.18:4807 , piece_id=OENCVZ7YL2MNBKFOWOQCVXADNZX3XLTJDWQHXWSSWRVVFKBQJ2UQ, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.482103780 (58h13m0.20466684s) [noise=true , address=77.90.31.18:4805 , piece_id=YKFCK64FI36JR7FWNAUISFLLXDYZPT4DFQN3WCQ3RMMFI4CZ5HTQ, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.482805443 (58h13m0.205368533s) [noise=true , address=104.36.82.11:4113 , piece_id=Z23NMSOTSZ3VQ2CNOY7VPBDPYEZAP2OO43XXS224X3VBHVYLTA3Q, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.483074607 (58h13m0.205637677s) [noise=true , address=45.91.116.8:4047 , piece_id=OEMRHEOQRBSOMOZ7CCLL2XFXQATXKMKGVZWL7FUZNRR5QSNJRC4A, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.483330728 (58h13m0.205893798s) [noise=true , address=45.91.116.9:4263 , piece_id=3BF57ZWMXHNKBEAF6OOPDGMHWO2KEL6WTBHNICNY4NV3LRNML4OQ, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.483717528 (58h13m0.206280578s) [noise=true , address=45.91.116.14:4191 , piece_id=43XDZUHX5VQXQV32MCK5IJ2KAWTN6NOKWCADFHWGUQ4CJTQKC5DA, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.483820244 (58h13m0.206383294s) [noise=true , address=45.91.116.6:4975 , piece_id=CYNWSV5BMOAWVOKY4VS7XWDW4AVXSZ7CPQ5WMTVNXL45BIGOCQXA, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.484361008 (58h13m0.206924048s) [noise=true , address=45.91.116.5:4299 , piece_id=42OWCOSUDO6KGUKD4ENSTRPL2NMFPL32TLCI27B56P4D2FXC7T5Q, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.484374805 (58h13m0.206937845s) [noise=true , address=104.36.82.5:4840 , piece_id=O4JYBINL5GQP2H3GOA6BO266TSHS7LKPBUZZ3BJYJUKCBVVGHPCA, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.484887345 (58h13m0.207450375s) [noise=true , address=45.91.116.10:4082 , piece_id=7WMQUTYIZEH5UG57BMTBR2C4GNVAT5UJJWA2XTG3GCUZREEC3XSA, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.484938183 (58h13m0.207501233s) [noise=true , address=45.91.116.4:4940 , piece_id=2DX2I4V2MFK2GKQBTV73HV3GRP4ST7YCQWHPYUNNRF7HCUZBVLEQ, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.485292160 (58h13m0.20785521s) [noise=true , address=45.91.116.11:4117 , piece_id=ZCEMFWRWHW4JWSQV33XT4N7RKO2FTCVXIQOMDTRIT3GTRSZWV5IA, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.485452597 (58h13m0.208015647s) [noise=true , address=45.91.116.4:4937 , piece_id=YFZNDI3HOU65RZFQT6IQVEKDS3QBQ4ZW6MQNWRCSN2ZDRA4B3NSQ, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.485942314 (58h13m0.208505364s) [noise=true , address=45.91.116.11:4119 , piece_id=CSDIN2S5DHIP2X33D3ZPGVX6KVDIWBRXPPNIX3JSGWQ3CL4PJIRQ, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:50.486346116 (58h13m0.208909156s) [noise=true , address=104.36.82.8:4972 , piece_id=3D3FH3R6LPNXLUHNLXIHJTCUI4DV2CHU5IL5RRJWMJYEWLSNBV3A, seg_pos={0 0}, upload=55871]: Uploading piece... Jul 26 21:48:52.210392193 (58h13m1.932955233s) [upload=55871]: Next returned nil segment Jul 26 21:48:52.210411891 (58h13m1.932974931s) [upload=55871]: Waiting for error group managing segments... Jul 26 21:48:52.486800517 (58h13m2.209363547s) [noise=true , address=45.91.116.4:4937 , piece_id=YFZNDI3HOU65RZFQT6IQVEKDS3QBQ4ZW6MQNWRCSN2ZDRA4B3NSQ, seg_pos={0 0}, upload=55871]: Done uploading piece. err: <nil> Jul 26 21:48:52.497125562 (58h13m2.219688592s) [noise=true , address=45.91.116.7:4009 , piece_id=L5NGIGVAFIEAZWXJ62L6CSWVQUOALVLXNZQOERUY6TT5JHX5VDGQ, seg_pos={0 0}, upload=55871]: Done uploading piece. err: <nil> Jul 26 21:48:52.508522808 (58h13m2.231085848s) [noise=true , address=45.91.116.11:4117 , piece_id=ZCEMFWRWHW4JWSQV33XT4N7RKO2FTCVXIQOMDTRIT3GTRSZWV5IA, seg_pos={0 0}, upload=55871]: Done uploading piece. err: <nil> Jul 26 21:48:52.510772703 (58h13m2.233335733s) [noise=true , address=45.91.116.9:4263 , piece_id=3BF57ZWMXHNKBEAF6OOPDGMHWO2KEL6WTBHNICNY4NV3LRNML4OQ, seg_pos={0 0}, upload=55871]: Done uploading piece. err: <nil> Jul 26 21:48:52.511651775 (58h13m2.234214815s) [noise=true , address=45.91.116.4:4940 , piece_id=2DX2I4V2MFK2GKQBTV73HV3GRP4ST7YCQWHPYUNNRF7HCUZBVLEQ, seg_pos={0 0}, upload=55871]: Done uploading piece. err: <nil> Jul 26 21:48:52.516233461 (58h13m2.238796501s) [noise=true , address=45.91.116.5:4299 , piece_id=42OWCOSUDO6KGUKD4ENSTRPL2NMFPL32TLCI27B56P4D2FXC7T5Q, seg_pos={0 0}, upload=55871]: Done uploading piece. err: <nil> Jul 26 21:48:52.563809529 (58h13m2.286372569s) [noise=true , address=45.91.116.8:4047 , piece_id=OEMRHEOQRBSOMOZ7CCLL2XFXQATXKMKGVZWL7FUZNRR5QSNJRC4A, seg_pos={0 0}, upload=55871]: Done uploading piece. err: <nil> Jul 26 21:48:52.593714386 (58h13m2.316277426s) [noise=true , address=45.91.116.6:4975 , piece_id=CYNWSV5BMOAWVOKY4VS7XWDW4AVXSZ7CPQ5WMTVNXL45BIGOCQXA, seg_pos={0 0}, upload=55871]: Done uploading piece. err: <nil> Jul 26 21:48:52.600372935 (58h13m2.322935975s) [noise=true , address=45.91.116.11:4119 , piece_id=CSDIN2S5DHIP2X33D3ZPGVX6KVDIWBRXPPNIX3JSGWQ3CL4PJIRQ, seg_pos={0 0}, upload=55871]: Done uploading piece. err: <nil> Jul 26 21:48:52.652449885 (58h13m2.375012915s) [noise=true , address=45.91.116.14:4191 , piece_id=43XDZUHX5VQXQV32MCK5IJ2KAWTN6NOKWCADFHWGUQ4CJTQKC5DA, seg_pos={0 0}, upload=55871]: Done uploading piece. err: <nil> Jul 26 21:48:52.661612025 (58h13m2.384175065s) [noise=true , address=45.91.116.10:4082 , piece_id=7WMQUTYIZEH5UG57BMTBR2C4GNVAT5UJJWA2XTG3GCUZREEC3XSA, seg_pos={0 0}, upload=55871]: Done uploading piece. err: <nil> Jul 26 21:49:19.601763512 (58h13m29.324326552s) [noise=true , address=77.90.31.18:4807 , piece_id=OENCVZ7YL2MNBKFOWOQCVXADNZX3XLTJDWQHXWSSWRVVFKBQJ2UQ, seg_pos={0 0}, upload=55871]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 21:49:19.601917677 (58h13m29.324480717s) [noise=true , address=104.36.82.8:4972 , piece_id=3D3FH3R6LPNXLUHNLXIHJTCUI4DV2CHU5IL5RRJWMJYEWLSNBV3A, seg_pos={0 0}, upload=55871]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: write tcp 188.166.225.170:36352->104.36.82.8:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:19.601968985 (58h13m29.324532025s) [noise=true , address=104.36.82.5:4840 , piece_id=O4JYBINL5GQP2H3GOA6BO266TSHS7LKPBUZZ3BJYJUKCBVVGHPCA, seg_pos={0 0}, upload=55871]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:104.36.82.5:4840): protocol: write tcp 188.166.225.170:33622->104.36.82.5:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:19.602035532 (58h13m29.324598572s) [noise=true , address=77.90.31.18:4805 , piece_id=YKFCK64FI36JR7FWNAUISFLLXDYZPT4DFQN3WCQ3RMMFI4CZ5HTQ, seg_pos={0 0}, upload=55871]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:57784->77.90.31.18:4805: use of closed network connection; write tcp 188.166.225.170:57784->77.90.31.18:4805: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57784->77.90.31.18:4805: use of closed network connection Jul 26 21:49:19.602158527 (58h13m29.324721567s) [noise=true , address=104.36.82.11:4113 , piece_id=Z23NMSOTSZ3VQ2CNOY7VPBDPYEZAP2OO43XXS224X3VBHVYLTA3Q, seg_pos={0 0}, upload=55871]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:104.36.82.11:4113): protocol: rpc: dial tcp 104.36.82.11:4113: i/o timeout; context deadline exceeded Jul 26 21:49:19.602188886 (58h13m29.324751926s) [seg_pos={0 0}, upload=55871]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 21:49:19.830624022 (58h13m29.553187062s) [upload=55871]: Done uploading segments... Jul 26 21:49:19.830651585 (58h13m29.553214625s) [upload=55871]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.230631831s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 20 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXXXXXXXXXXX] duration: 1m0.235469194s FAIL (2025-07-26 21:49:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== piece_id=3LFZOXIQKK33M2BU6L26JPSZREE2CJYW6KDNYRMSZ4GJBODO6UMA, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:01.122228842 (58h12m10.844791872s) [noise=true , address=104.36.82.55:4224 , piece_id=7CIUX7ZONHNYLVANFJREKSIYRROQHFKIJG52G76ZRE5DJE3BLD5Q, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:01.122259741 (58h12m10.844822781s) [noise=true , address=104.36.82.12:4182 , piece_id=SDVTELAN7OHF2Q6ZJMDNQQPKQGCUSPDEHUTFK2PMLLMY2G2C4LQA, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:01.122471125 (58h12m10.845034155s) [noise=true , address=45.91.116.14:4222 , piece_id=LIME3UPW6WQ73VMLADFJ7N5BAUMORFGQ7LUNJ5MMB7KOTC7PKOIQ, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:01.122476165 (58h12m10.845039205s) [noise=true , address=104.36.82.54:4183 , piece_id=KGZGKWQOCKCITKCUNYTMDTCFCFW74CENDYNXSJH57HOKQULXAIPA, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:01.122776579 (58h12m10.845339619s) [noise=true , address=156.146.43.225:4433 , piece_id=U7JFUHFEZUNL5LNJED4JMMYLGOBLTT3XEAKN7URV543DP3NLCXKA, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:01.123003754 (58h12m10.845566794s) [noise=true , address=104.36.82.46:4482 , piece_id=4DXDEF4CQUOW4SG6TEJDP3W3HB7363WMIIKRFJ3B462K62PBGA4A, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:01.123198868 (58h12m10.845761908s) [noise=true , address=104.36.82.47:4522 , piece_id=C62EQHOVRZ4DNUFUODQJE2YJG7AZKFHLP4KIQM4PGGVXLP2CID3Q, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:01.123392538 (58h12m10.845955578s) [noise=true , address=104.36.82.50:4018 , piece_id=BGRV5IM2A5D7J7ORTNKGBYTIRZVTKZVXNUXGDXI237RPFBGCSBZA, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:01.123593843 (58h12m10.846156883s) [noise=true , address=104.36.82.51:4055 , piece_id=YP24RCGBDE5IV6QR22ZSCUSIUJDAKGSETP36ZI36VOIAIT5YTLYQ, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:01.123813343 (58h12m10.846376383s) [noise=true , address=104.36.82.3:4799 , piece_id=M4PGJWPQVO37CQWQ6HE7HS4DNUZV4XOQ2Q24PWDYV6MIFY4JOMBQ, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:01.124004839 (58h12m10.846567879s) [noise=true , address=104.36.82.43:4865 , piece_id=X7SMNTBFRLIGERNCPMYAND6EW7OX5IKEKUZK32N3QSM6J73V7EWQ, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:01.124896536 (58h12m10.847459576s) [noise=true , address=104.36.82.53:4153 , piece_id=WJR3S5KTWLIYLIAVHODCINUI67RW6DVGXKGSY7VNPSGFVAXHV7PQ, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:01.128897150 (58h12m10.85146019s) [noise=true , address=77.90.31.17:4786 , piece_id=HSF3UEIZB2QOCMOWM3CWEVNWDIQJT6LY644RATYJSSE2RF4NCIUA, seg_pos={0 0}, upload=55859]: Uploading piece... Jul 26 21:48:02.212522576 (58h12m11.935085616s) [upload=55859]: Next returned nil segment Jul 26 21:48:02.212539940 (58h12m11.93510298s) [upload=55859]: Waiting for error group managing segments... Jul 26 21:48:02.729243401 (58h12m12.451806431s) [noise=true , address=79.127.205.245:4244 , piece_id=PZIXW4OW73DUVTHSDSIR3GNPTZMKOGG6VLIH7ZRYUEDQEEJD2DMA, seg_pos={0 0}, upload=55859]: Done uploading piece. err: <nil> Jul 26 21:48:03.097579021 (58h12m12.820142061s) [noise=true , address=79.127.219.38:4005 , piece_id=3LFZOXIQKK33M2BU6L26JPSZREE2CJYW6KDNYRMSZ4GJBODO6UMA, seg_pos={0 0}, upload=55859]: Done uploading piece. err: <nil> Jul 26 21:48:03.342489078 (58h12m13.065052118s) [noise=true , address=45.91.116.14:4222 , piece_id=LIME3UPW6WQ73VMLADFJ7N5BAUMORFGQ7LUNJ5MMB7KOTC7PKOIQ, seg_pos={0 0}, upload=55859]: Done uploading piece. err: <nil> Jul 26 21:48:06.121563136 (58h12m15.844126166s) [noise=true , address=156.146.43.225:4433 , piece_id=U7JFUHFEZUNL5LNJED4JMMYLGOBLTT3XEAKN7URV543DP3NLCXKA, seg_pos={0 0}, upload=55859]: Done uploading piece. err: <nil> Jul 26 21:48:06.677011319 (58h12m16.399574359s) [noise=true , address=79.127.201.215:4343 , piece_id=4R66XHCCPM5ASKU2UQVJQLI33CZCHOJWJUCDREKJOABEGE4N3RHA, seg_pos={0 0}, upload=55859]: Done uploading piece. err: <nil> Jul 26 21:48:09.386060146 (58h12m19.108623186s) [noise=true , address=77.90.31.17:4786 , piece_id=HSF3UEIZB2QOCMOWM3CWEVNWDIQJT6LY644RATYJSSE2RF4NCIUA, seg_pos={0 0}, upload=55859]: Done uploading piece. err: <nil> Jul 26 21:49:00.032331284 (58h13m9.754894324s) [noise=true , address=104.36.82.8:4981 , piece_id=IQXRYQND5V7LK3W2QISLXWV33AQE6OBOO7IGUD2UGEFZPFACBDLA, seg_pos={0 0}, upload=55859]: Done uploading piece. err: ecclient: upload failed (node:12aNMDfQppoRyZNge9pD93mMHGJ2JJu7T39UWBPSXavUwqnCxQM, address:104.36.82.8:4981): protocol: rpc: dial tcp 104.36.82.8:4981: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:00.032374576 (58h13m9.754937616s) [noise=true , address=104.36.82.46:4482 , piece_id=4DXDEF4CQUOW4SG6TEJDP3W3HB7363WMIIKRFJ3B462K62PBGA4A, seg_pos={0 0}, upload=55859]: Done uploading piece. err: ecclient: upload failed (node:12mNCKAdMzWu87fmyrcwDKZvscMcu2CogjXicwAi5wLSFJa7oVV, address:104.36.82.46:4482): protocol: rpc: dial tcp 104.36.82.46:4482: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:00.032409554 (58h13m9.754972584s) [noise=true , address=104.36.82.52:4085 , piece_id=QB3R3BKUIJ5Q43EGT7L3YAUQ2UWSLAO6CKWENF5ZFY4RKGMUBFSA, seg_pos={0 0}, upload=55859]: Done uploading piece. err: ecclient: upload failed (node:17x7QtnZsENMZPU3tqkh8jEd29JdowLvT855W4kuourgUA6y5C, address:104.36.82.52:4085): protocol: rpc: dial tcp 104.36.82.52:4085: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:00.032441034 (58h13m9.755004074s) [noise=true , address=104.36.82.55:4224 , piece_id=7CIUX7ZONHNYLVANFJREKSIYRROQHFKIJG52G76ZRE5DJE3BLD5Q, seg_pos={0 0}, upload=55859]: Done uploading piece. err: ecclient: upload failed (node:12EjKmUiv4pxkVCB4WoiXim4uFpxrBPyfiSS4efTY3HKSE3jb5y, address:104.36.82.55:4224): protocol: rpc: dial tcp 104.36.82.55:4224: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:00.032471602 (58h13m9.755034632s) [noise=true , address=104.36.82.12:4182 , piece_id=SDVTELAN7OHF2Q6ZJMDNQQPKQGCUSPDEHUTFK2PMLLMY2G2C4LQA, seg_pos={0 0}, upload=55859]: Done uploading piece. err: ecclient: upload failed (node:1KSPtVhJPKYAb3Gj25ihyExNgkDCGo9QTDaFcEfXRfLdDz8nsh, address:104.36.82.12:4182): protocol: rpc: dial tcp 104.36.82.12:4182: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:00.032500056 (58h13m9.755063096s) [noise=true , address=104.36.82.50:4018 , piece_id=BGRV5IM2A5D7J7ORTNKGBYTIRZVTKZVXNUXGDXI237RPFBGCSBZA, seg_pos={0 0}, upload=55859]: Done uploading piece. err: ecclient: upload failed (node:12YcYgDWq2k7F9fwpdLAGNkrskGEUURe6QuRed7mRjMTdNHEQ8D, address:104.36.82.50:4018): protocol: rpc: dial tcp 104.36.82.50:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:00.032527048 (58h13m9.755090088s) [noise=true , address=104.36.82.51:4055 , piece_id=YP24RCGBDE5IV6QR22ZSCUSIUJDAKGSETP36ZI36VOIAIT5YTLYQ, seg_pos={0 0}, upload=55859]: Done uploading piece. err: ecclient: upload failed (node:1SAWKJT2JEb9ReJUGDZ545Rp7LeQoCJKYsQgmrDA9csufPf8us, address:104.36.82.51:4055): protocol: rpc: dial tcp 104.36.82.51:4055: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:00.032556154 (58h13m9.755119194s) [noise=true , address=104.36.82.3:4799 , piece_id=M4PGJWPQVO37CQWQ6HE7HS4DNUZV4XOQ2Q24PWDYV6MIFY4JOMBQ, seg_pos={0 0}, upload=55859]: Done uploading piece. err: ecclient: upload failed (node:12pMB9xsz6ArzVRzSkh59X8Y7v3w18CzWoVzBRhPdKZTne241v9, address:104.36.82.3:4799): protocol: rpc: dial tcp 104.36.82.3:4799: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:00.032581993 (58h13m9.755145033s) [noise=true , address=104.36.82.43:4865 , piece_id=X7SMNTBFRLIGERNCPMYAND6EW7OX5IKEKUZK32N3QSM6J73V7EWQ, seg_pos={0 0}, upload=55859]: Done uploading piece. err: ecclient: upload failed (node:12DQ2GHBBNrbKsE6eo3YTsMkQGuekVYwrQQjBdQWtQWNgd35VPQ, address:104.36.82.43:4865): protocol: rpc: dial tcp 104.36.82.43:4865: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:00.032609686 (58h13m9.755172726s) [noise=true , address=104.36.82.53:4153 , piece_id=WJR3S5KTWLIYLIAVHODCINUI67RW6DVGXKGSY7VNPSGFVAXHV7PQ, seg_pos={0 0}, upload=55859]: Done uploading piece. err: ecclient: upload failed (node:1fPp4DvLe18dZWPmMPFiC8o9jL4uiEFNmPAHHmFEH74ZXNndw3, address:104.36.82.53:4153): protocol: rpc: dial tcp 104.36.82.53:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:00.032634814 (58h13m9.755197854s) [noise=true , address=104.36.82.54:4183 , piece_id=KGZGKWQOCKCITKCUNYTMDTCFCFW74CENDYNXSJH57HOKQULXAIPA, seg_pos={0 0}, upload=55859]: Done uploading piece. err: ecclient: upload failed (node:1kovWN9SiUhcGQtZitjH9zgPj7iHaZwY5v37bjwPvArmgX44jA, address:104.36.82.54:4183): protocol: rpc: dial tcp 104.36.82.54:4183: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:00.032663910 (58h13m9.75522694s) [noise=true , address=104.36.82.47:4522 , piece_id=C62EQHOVRZ4DNUFUODQJE2YJG7AZKFHLP4KIQM4PGGVXLP2CID3Q, seg_pos={0 0}, upload=55859]: Done uploading piece. err: ecclient: upload failed (node:1zGZe6e37uop4y622fU1GbspMPVEZURS3FbMrf7je6VT9TYGFr, address:104.36.82.47:4522): protocol: write tcp 188.166.225.170:55102->104.36.82.47:4522: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:49:00.032676614 (58h13m9.755239654s) [seg_pos={0 0}, upload=55859]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Jul 26 21:49:00.265388563 (58h13m9.987951603s) [upload=55859]: Done uploading segments... Jul 26 21:49:00.265415885 (58h13m9.987978925s) [upload=55859]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 1m0.23548247s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [............X.........X.................X...................] duration: 1.856038239s FAIL (2025-07-26 21:48:56 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.856076993s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................................X....X......XX....XX.XX.X] duration: 30.243249627s FAIL (2025-07-26 21:48:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 845940046 (58h11m58.568503076s) [seg_pos={0 0}, upload=55854]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:47:48.845998718 (58h11m58.568561758s) [upload=55854]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:47:48.846002576 (58h11m58.568565616s) [upload=55854]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:47:49.140928740 (58h11m58.86349177s) [upload=55854]: Done uploading segments... Jul 26 21:47:49.140956453 (58h11m58.863519513s) [upload=55854]: Upload finished. err: <nil> Jul 26 21:48:19.766192356 (58h12m29.488755396s) [upload=55863]: Starting upload Jul 26 21:48:19.766244946 (58h12m29.488807987s) [upload=55863]: Done starting upload Jul 26 21:48:19.766335070 (58h12m29.48889811s) [upload=55863]: Uploading segments... Jul 26 21:48:19.766416055 (58h12m29.488979095s) [upload=55863]: Got next segment. Inline: false Jul 26 21:48:19.766440450 (58h12m29.48900349s) [upload=55863]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:48:19.766444940 (58h12m29.48900798s) [upload=55863]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:48:20.001071753 (58h12m29.723634803s) [seg_pos={0 0}, upload=55863]: Begin upload segment... Jul 26 21:48:20.001113362 (58h12m29.723676392s) [seg_pos={0 0}, upload=55863]: Done begin upload segment. Jul 26 21:48:20.001137108 (58h12m29.723700138s) [noise=true , address=77.90.31.17:4771 , piece_id=EWMOLBMO73OVO2MLA5BJUEP64R7ISMVJQ7TABPUCEDXWRCFCADRA, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.001140825 (58h12m29.723703935s) [noise=true , address=45.91.116.12:4226 , piece_id=S5UH6J2IXU5U47EHES4SWBXJTNBW5VDOIYIFUMBC2VCZ6IY3HSEA, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.001486066 (58h12m29.724049106s) [noise=true , address=45.91.116.11:4117 , piece_id=F7TRSIGWJ6Y5NG2RHOSCMXRNU7KDZMXMBEUKV6LVPIEVJFLCRQBA, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.001664948 (58h12m29.724227988s) [noise=true , address=45.91.116.4:4939 , piece_id=AAS4H3ZHQX7P4VAAHBRNO734M2T65D3C2UBS6GHZC636U6WZJNWA, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.001733339 (58h12m29.724296379s) [noise=true , address=45.91.116.12:4225 , piece_id=AARLFUMU5UXEWIAQ2XH4TWDATM3GXJXNOKJTO7ICRIILNMM7VQVA, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.002028273 (58h12m29.724591303s) [noise=true , address=45.91.116.7:4010 , piece_id=NPRFMKOWH3QPGBQBTAWC3ZY3US2PP2RXLVNG5R4R4OZR3RDSO5ZQ, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.002319089 (58h12m29.724882119s) [noise=true , address=45.91.116.14:4190 , piece_id=VJUM2G5O7KJINHZPHIT3BPKAUPX6CNFQNAYFS7KAGHBETJHCSSKQ, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.002342384 (58h12m29.724905424s) [noise=true , address=104.36.82.5:4839 , piece_id=4GDV4RINXENDB7I5NVOL3ZWN5XHF2LSZDT4I5MNAYGWTJHESQVXA, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.002520004 (58h12m29.725083044s) [noise=true , address=77.90.31.15:4698 , piece_id=UHZVCFXGIZQ56QJ5T2CUOGEVKY3W447PKJTKHHENS67CMZDFU4QA, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.002736257 (58h12m29.725299297s) [noise=true , address=104.36.82.14:4222 , piece_id=REUZKN42ZPKNHERNRTO4ZI734QA4FTIYZ5W3A423DF5Q4USOUJXA, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.002950998 (58h12m29.725514039s) [noise=true , address=104.36.82.4:4803 , piece_id=YQNCCLDC5P73T3L3ROSCWUXTPUYOPSEKBSUO62M5JX47QRKVHCEA, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.003194905 (58h12m29.725757945s) [noise=true , address=45.91.116.9:4261 , piece_id=OIFUP6AFHP47AWCLL6X44AEMXLCTBIZZMACHIJLPWCULI3OK5WWA, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.003409446 (58h12m29.725972486s) [noise=true , address=77.90.31.12:4590 , piece_id=NV4QXNB7EPNQMZMRRTQZU7NMBMK55N4DJ2W7ZY6X2X24SV3GK5BQ, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.003614848 (58h12m29.726177889s) [noise=true , address=45.91.116.8:4047 , piece_id=45OXDNPOVRPYYCUYIS7Z4N2EZY6MFXAAOK5QDPH2RYWAWSTRWEIA, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.003794252 (58h12m29.726357292s) [noise=true , address=45.91.116.13:4155 , piece_id=SEBPLHNKDOHVGPTEVS3RGFWKMENKHDJ2A3KQET5TD5SW64XNRW3Q, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:20.003978464 (58h12m29.726541504s) [noise=true , address=45.91.116.6:4973 , piece_id=ORIFRKR4VDPE5PBUOLGGR7EBABLTQN3KLF2LKJDMPH7BV7276OJQ, seg_pos={0 0}, upload=55863]: Uploading piece... Jul 26 21:48:21.729716878 (58h12m31.452279908s) [upload=55863]: Next returned nil segment Jul 26 21:48:21.729738099 (58h12m31.452301129s) [upload=55863]: Waiting for error group managing segments... Jul 26 21:48:22.022543335 (58h12m31.745106375s) [noise=true , address=45.91.116.4:4939 , piece_id=AAS4H3ZHQX7P4VAAHBRNO734M2T65D3C2UBS6GHZC636U6WZJNWA, seg_pos={0 0}, upload=55863]: Done uploading piece. err: <nil> Jul 26 21:48:22.028789545 (58h12m31.751352575s) [noise=true , address=45.91.116.13:4155 , piece_id=SEBPLHNKDOHVGPTEVS3RGFWKMENKHDJ2A3KQET5TD5SW64XNRW3Q, seg_pos={0 0}, upload=55863]: Done uploading piece. err: <nil> Jul 26 21:48:22.029977869 (58h12m31.752540919s) [noise=true , address=45.91.116.12:4226 , piece_id=S5UH6J2IXU5U47EHES4SWBXJTNBW5VDOIYIFUMBC2VCZ6IY3HSEA, seg_pos={0 0}, upload=55863]: Done uploading piece. err: <nil> Jul 26 21:48:22.032135648 (58h12m31.754698688s) [noise=true , address=45.91.116.11:4117 , piece_id=F7TRSIGWJ6Y5NG2RHOSCMXRNU7KDZMXMBEUKV6LVPIEVJFLCRQBA, seg_pos={0 0}, upload=55863]: Done uploading piece. err: <nil> Jul 26 21:48:22.043692850 (58h12m31.76625589s) [noise=true , address=45.91.116.7:4010 , piece_id=NPRFMKOWH3QPGBQBTAWC3ZY3US2PP2RXLVNG5R4R4OZR3RDSO5ZQ, seg_pos={0 0}, upload=55863]: Done uploading piece. err: <nil> Jul 26 21:48:22.077020766 (58h12m31.799583806s) [noise=true , address=45.91.116.8:4047 , piece_id=45OXDNPOVRPYYCUYIS7Z4N2EZY6MFXAAOK5QDPH2RYWAWSTRWEIA, seg_pos={0 0}, upload=55863]: Done uploading piece. err: <nil> Jul 26 21:48:22.116401784 (58h12m31.838964825s) [noise=true , address=45.91.116.12:4225 , piece_id=AARLFUMU5UXEWIAQ2XH4TWDATM3GXJXNOKJTO7ICRIILNMM7VQVA, seg_pos={0 0}, upload=55863]: Done uploading piece. err: <nil> Jul 26 21:48:22.172727062 (58h12m31.895290102s) [noise=true , address=45.91.116.9:4261 , piece_id=OIFUP6AFHP47AWCLL6X44AEMXLCTBIZZMACHIJLPWCULI3OK5WWA, seg_pos={0 0}, upload=55863]: Done uploading piece. err: <nil> Jul 26 21:48:22.182658434 (58h12m31.905221474s) [noise=true , address=45.91.116.14:4190 , piece_id=VJUM2G5O7KJINHZPHIT3BPKAUPX6CNFQNAYFS7KAGHBETJHCSSKQ, seg_pos={0 0}, upload=55863]: Done uploading piece. err: <nil> Jul 26 21:48:22.183255817 (58h12m31.905818857s) [noise=true , address=45.91.116.6:4973 , piece_id=ORIFRKR4VDPE5PBUOLGGR7EBABLTQN3KLF2LKJDMPH7BV7276OJQ, seg_pos={0 0}, upload=55863]: Done uploading piece. err: <nil> Jul 26 21:48:35.832730126 (58h12m45.555293156s) [noise=true , address=77.90.31.17:4771 , piece_id=EWMOLBMO73OVO2MLA5BJUEP64R7ISMVJQ7TABPUCEDXWRCFCADRA, seg_pos={0 0}, upload=55863]: Done uploading piece. err: <nil> Jul 26 21:48:49.118348830 (58h12m58.84091186s) [noise=true , address=77.90.31.15:4698 , piece_id=UHZVCFXGIZQ56QJ5T2CUOGEVKY3W447PKJTKHHENS67CMZDFU4QA, seg_pos={0 0}, upload=55863]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 21:48:49.118581306 (58h12m58.841144336s) [noise=true , address=77.90.31.12:4590 , piece_id=NV4QXNB7EPNQMZMRRTQZU7NMBMK55N4DJ2W7ZY6X2X24SV3GK5BQ, seg_pos={0 0}, upload=55863]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:59168->77.90.31.12:4590: use of closed network connection; write tcp 188.166.225.170:59168->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59168->77.90.31.12:4590: use of closed network connection Jul 26 21:48:49.118620931 (58h12m58.841183971s) [noise=true , address=104.36.82.4:4803 , piece_id=YQNCCLDC5P73T3L3ROSCWUXTPUYOPSEKBSUO62M5JX47QRKVHCEA, seg_pos={0 0}, upload=55863]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:104.36.82.4:4803): protocol: write tcp 188.166.225.170:38306->104.36.82.4:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:48:49.118654906 (58h12m58.841217946s) [noise=true , address=104.36.82.5:4839 , piece_id=4GDV4RINXENDB7I5NVOL3ZWN5XHF2LSZDT4I5MNAYGWTJHESQVXA, seg_pos={0 0}, upload=55863]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:104.36.82.5:4839): protocol: write tcp 188.166.225.170:42736->104.36.82.5:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:48:49.118660456 (58h12m58.841223496s) [noise=true , address=104.36.82.14:4222 , piece_id=REUZKN42ZPKNHERNRTO4ZI734QA4FTIYZ5W3A423DF5Q4USOUJXA, seg_pos={0 0}, upload=55863]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: rpc: dial tcp 104.36.82.14:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:48:49.118680675 (58h12m58.841243715s) [seg_pos={0 0}, upload=55863]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 21:48:49.359136580 (58h12m59.08169966s) [upload=55863]: Done uploading segments... Jul 26 21:48:49.359164624 (58h12m59.081727694s) [upload=55863]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.243274334s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.............X.........X.................X..................] duration: 1.84966789s FAIL (2025-07-26 21:48:31 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.84969431s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [.....X...................................XX.........XX.XX.XX] duration: 30.262644489s FAIL (2025-07-26 21:48:26 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.26266099s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration: 1m0.252155723s FAIL (2025-07-26 21:47:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== 6 , piece_id=6UMUFHQVVS2SU3TTAPTAV5HLOSVU5EVMIJ3ZM2HL44F5QSNJKEJQ, seg_pos={0 0}, upload=55843]: Uploading piece... Jul 26 21:47:00.650639381 (58h11m10.373202411s) [noise=true , address=104.36.82.35:4590 , piece_id=YH7CBNOZE2UHNSIJKVPPADVXZRQ7TZB6MLNAC3MHFKYBKQ334IQQ, seg_pos={0 0}, upload=55843]: Uploading piece... Jul 26 21:47:00.650800199 (58h11m10.373363239s) [noise=true , address=104.36.82.20:4449 , piece_id=XGAOIGKUDHPO6VI2DQI2EUSURAC7EWL4QTGUSE7WYH33QTX3ZLPQ, seg_pos={0 0}, upload=55843]: Uploading piece... Jul 26 21:47:00.650901774 (58h11m10.373464814s) [noise=true , address=104.36.82.5:4852 , piece_id=LG4NWNPCEBBQ3XE2Q6CWE2HHNYTFFAHV5AJBJLK3KFYC3RHLHA2A, seg_pos={0 0}, upload=55843]: Uploading piece... Jul 26 21:47:00.651036692 (58h11m10.373599732s) [noise=true , address=79.127.205.234:4610 , piece_id=MHZVKJDYEM46YSL5ZMPGJ2M5QUMFNMOQESW3NIVVQZ3Y7J3X6JPQ, seg_pos={0 0}, upload=55843]: Uploading piece... Jul 26 21:47:00.651256102 (58h11m10.373819142s) [noise=true , address=79.127.205.233:4655 , piece_id=5RESNLS2Q3YUQSRF6LLJF77N2ZYXJDCCDDTWC653BCXLB3XZN7XA, seg_pos={0 0}, upload=55843]: Uploading piece... Jul 26 21:47:00.651465141 (58h11m10.374028181s) [noise=true , address=104.36.82.46:4500 , piece_id=X5YCJ5Y6OQLXCL3CQQATMRTC3REMJWHLEBZR6Q7R7GWIGHSG7ZGQ, seg_pos={0 0}, upload=55843]: Uploading piece... Jul 26 21:47:00.651770155 (58h11m10.374333185s) [noise=true , address=104.36.82.55:4208 , piece_id=OGLB5W3Z7TUFDFH5N7QZ4Y2G6RBIEPT3CPU4BVWEC4JBJ6H35YFQ, seg_pos={0 0}, upload=55843]: Uploading piece... Jul 26 21:47:00.652004163 (58h11m10.374567193s) [noise=true , address=104.36.82.52:4104 , piece_id=CI6ZJ2BSGNW2V4LRE4U44BKS2CSQGQNUGFN7PYK7R24UWZ3DQLYQ, seg_pos={0 0}, upload=55843]: Uploading piece... Jul 26 21:47:00.652270894 (58h11m10.374833934s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBHFDK4YM2LBUDVH4IYTR7VQJZTOHITVRGIITFPMIOYCF5GCFSA, seg_pos={0 0}, upload=55843]: Uploading piece... Jul 26 21:47:00.652532413 (58h11m10.375095453s) [noise=true , address=104.36.82.56:4255 , piece_id=CQMGO2WPIZVSC5YAL7RKMDEYWY5FNRXYZ5GIZJA3OIT7OFSI4LWQ, seg_pos={0 0}, upload=55843]: Uploading piece... Jul 26 21:47:00.652765440 (58h11m10.37532848s) [noise=true , address=104.36.82.43:4861 , piece_id=MCG7TVK6VY6M4HBRYFNI7NWP5JBUAQTMUYOHA2MQICTLHKVA7ZZQ, seg_pos={0 0}, upload=55843]: Uploading piece... Jul 26 21:47:01.741589473 (58h11m11.464152513s) [upload=55843]: Next returned nil segment Jul 26 21:47:01.741607767 (58h11m11.464170807s) [upload=55843]: Waiting for error group managing segments... Jul 26 21:47:02.041482409 (58h11m11.764045459s) [noise=true , address=79.127.205.246:4280 , piece_id=FLBHFDK4YM2LBUDVH4IYTR7VQJZTOHITVRGIITFPMIOYCF5GCFSA, seg_pos={0 0}, upload=55843]: Done uploading piece. err: <nil> Jul 26 21:47:02.069982499 (58h11m11.792545539s) [noise=true , address=79.127.205.245:4248 , piece_id=JG46OOJOHMXRXUATDQKV7SLPV55LX237LUE3IEFYI7AWUD6BL7EQ, seg_pos={0 0}, upload=55843]: Done uploading piece. err: <nil> Jul 26 21:47:02.122613407 (58h11m11.845176448s) [noise=true , address=79.127.205.234:4610 , piece_id=MHZVKJDYEM46YSL5ZMPGJ2M5QUMFNMOQESW3NIVVQZ3Y7J3X6JPQ, seg_pos={0 0}, upload=55843]: Done uploading piece. err: <nil> Jul 26 21:47:02.146568902 (58h11m11.869131972s) [noise=true , address=79.127.205.233:4655 , piece_id=5RESNLS2Q3YUQSRF6LLJF77N2ZYXJDCCDDTWC653BCXLB3XZN7XA, seg_pos={0 0}, upload=55843]: Done uploading piece. err: <nil> Jul 26 21:47:02.460465241 (58h11m12.183028281s) [noise=true , address=79.127.220.101:4048 , piece_id=KR4NCKDDYGDLEH5J3VTPHLSYZFMT2MDJ5YKNKLEVXLU5FIRVSLNA, seg_pos={0 0}, upload=55843]: Done uploading piece. err: <nil> Jul 26 21:47:03.948843032 (58h11m13.671406072s) [noise=true , address=79.127.201.215:4343 , piece_id=2JOOCSEPFJ75WXPJUTYDJL2EAXBOJBYOO4TTFWHO4F5YWFONGGCQ, seg_pos={0 0}, upload=55843]: Done uploading piece. err: <nil> Jul 26 21:47:59.546708940 (58h12m9.26927197s) [noise=true , address=104.36.82.53:4136 , piece_id=6UMUFHQVVS2SU3TTAPTAV5HLOSVU5EVMIJ3ZM2HL44F5QSNJKEJQ, seg_pos={0 0}, upload=55843]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:104.36.82.53:4136): protocol: write tcp 188.166.225.170:33390->104.36.82.53:4136: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:47:59.546779125 (58h12m9.269342165s) [noise=true , address=104.36.82.54:4183 , piece_id=CP72BWAZOEWMY62Q3OXC3YWC2AHBJHQLLRENETRSCREII7CHJU6A, seg_pos={0 0}, upload=55843]: Done uploading piece. err: ecclient: upload failed (node:1kovWN9SiUhcGQtZitjH9zgPj7iHaZwY5v37bjwPvArmgX44jA, address:104.36.82.54:4183): protocol: write tcp 188.166.225.170:50390->104.36.82.54:4183: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:47:59.546891340 (58h12m9.26945438s) [noise=true , address=104.36.82.56:4255 , piece_id=CQMGO2WPIZVSC5YAL7RKMDEYWY5FNRXYZ5GIZJA3OIT7OFSI4LWQ, seg_pos={0 0}, upload=55843]: Done uploading piece. err: ecclient: upload failed (node:12S6rwimXBvaEb4KneAwL5H3t5zTmvGVhyJSWCTU1u17r4RWJta, address:104.36.82.56:4255): protocol: write tcp 188.166.225.170:55116->104.36.82.56:4255: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:47:59.546955382 (58h12m9.269518422s) [noise=true , address=104.36.82.35:4590 , piece_id=YH7CBNOZE2UHNSIJKVPPADVXZRQ7TZB6MLNAC3MHFKYBKQ334IQQ, seg_pos={0 0}, upload=55843]: Done uploading piece. err: ecclient: upload failed (node:128t6B6PhCQznqwMTGJdkWuuCyqQ4bVHbPbZWWnYwb8Eyc1DjDw, address:104.36.82.35:4590): protocol: rpc: dial tcp 104.36.82.35:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:47:59.547000999 (58h12m9.26956403s) [noise=true , address=104.36.82.20:4449 , piece_id=XGAOIGKUDHPO6VI2DQI2EUSURAC7EWL4QTGUSE7WYH33QTX3ZLPQ, seg_pos={0 0}, upload=55843]: Done uploading piece. err: ecclient: upload failed (node:1r53UYADNA4XuUzpWdAvUh6T3iUJSbX9YvqnimzoPTgMfSmBWF, address:104.36.82.20:4449): protocol: write tcp 188.166.225.170:42336->104.36.82.20:4449: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:47:59.547078076 (58h12m9.269641106s) [noise=true , address=104.36.82.5:4852 , piece_id=LG4NWNPCEBBQ3XE2Q6CWE2HHNYTFFAHV5AJBJLK3KFYC3RHLHA2A, seg_pos={0 0}, upload=55843]: Done uploading piece. err: ecclient: upload failed (node:12sQvAZS92R7TTJUFjMRieYT6BP4vp2UAan4PHrhPt6m3eHPukN, address:104.36.82.5:4852): protocol: write tcp 188.166.225.170:45256->104.36.82.5:4852: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:47:59.547123574 (58h12m9.269686614s) [noise=true , address=104.36.82.46:4500 , piece_id=X5YCJ5Y6OQLXCL3CQQATMRTC3REMJWHLEBZR6Q7R7GWIGHSG7ZGQ, seg_pos={0 0}, upload=55843]: Done uploading piece. err: ecclient: upload failed (node:12B7ox4ehqxJ2mwqeXj8C9N9GuSU5etNC23iT1pJnTgMLGBAGQK, address:104.36.82.46:4500): protocol: write tcp 188.166.225.170:46100->104.36.82.46:4500: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:47:59.547155374 (58h12m9.269718414s) [noise=true , address=104.36.82.55:4208 , piece_id=OGLB5W3Z7TUFDFH5N7QZ4Y2G6RBIEPT3CPU4BVWEC4JBJ6H35YFQ, seg_pos={0 0}, upload=55843]: Done uploading piece. err: ecclient: upload failed (node:12pTsKDUmbk8xYNnjrcYyX4qxwLSfHzsGgkUGSbrBiWPARfBrfi, address:104.36.82.55:4208): protocol: write tcp 188.166.225.170:57252->104.36.82.55:4208: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:47:59.547209949 (58h12m9.269772989s) [noise=true , address=104.36.82.43:4861 , piece_id=MCG7TVK6VY6M4HBRYFNI7NWP5JBUAQTMUYOHA2MQICTLHKVA7ZZQ, seg_pos={0 0}, upload=55843]: Done uploading piece. err: ecclient: upload failed (node:122HC6BetMaBDNGEaHVwmadWtV1nCHx3vGBK2SWVDEQS1iBCz4o, address:104.36.82.43:4861): protocol: write tcp 188.166.225.170:42706->104.36.82.43:4861: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:47:59.547265485 (58h12m9.269828515s) [noise=true , address=104.36.82.34:4566 , piece_id=UPTESAKII6NHNMT4KHLIEXBPD6LHQP3ELS37Z3R4NVAQU4TMU6KA, seg_pos={0 0}, upload=55843]: Done uploading piece. err: ecclient: upload failed (node:1Z9ic1vvfDViKVZbrkLZnryBxKKQ4rJdjBFY6wEYCDtJt9ci77, address:104.36.82.34:4566): protocol: write tcp 188.166.225.170:59852->104.36.82.34:4566: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:47:59.547328375 (58h12m9.269891405s) [noise=true , address=104.36.82.42:4806 , piece_id=4BKFCCDIQLRTCRQ2OX7FBXG66QDWO5YAXP3GFEPKVPMLKK5BRT3A, seg_pos={0 0}, upload=55843]: Done uploading piece. err: ecclient: upload failed (node:1tWnxuTSo4yHAJUzvVFTmwPSNb8cBJJJ17BnKd5Z3HMYf66vSC, address:104.36.82.42:4806): protocol: write tcp 188.166.225.170:43872->104.36.82.42:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:47:59.547398379 (58h12m9.269961419s) [noise=true , address=104.36.82.52:4104 , piece_id=CI6ZJ2BSGNW2V4LRE4U44BKS2CSQGQNUGFN7PYK7R24UWZ3DQLYQ, seg_pos={0 0}, upload=55843]: Done uploading piece. err: ecclient: upload failed (node:12qmPpCmXAvuJZn2KR5Kr4nHYwKEzWvwqZ82nqNka5u1mKaZmXs, address:104.36.82.52:4104): protocol: write tcp 188.166.225.170:43692->104.36.82.52:4104: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:47:59.547416154 (58h12m9.269979194s) [seg_pos={0 0}, upload=55843]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Jul 26 21:47:59.797297172 (58h12m9.519860202s) [upload=55843]: Done uploading segments... Jul 26 21:47:59.797323332 (58h12m9.519886372s) [upload=55843]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 1m0.25216945s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [......X...................................XX.........XX.XX.X] duration: 30.238747275s FAIL (2025-07-26 21:47:56 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.238770288s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............X.........X.................X.................] duration: 1.838215307s FAIL (2025-07-26 21:47:56 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.838240886s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....................................X....X......XX....XX.XX.] duration: 14.186561303s FAIL (2025-07-26 21:47:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== C4WXBJRYY33IBBO6G2PKD6UHQYE73D7BEJA, seg_pos={0 0}, upload=55830]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:104.36.82.7:4912): protocol: rpc: dial tcp 104.36.82.7:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:32.069210546 (58h10m41.791773586s) [noise=true , address=104.36.82.4:4802 , piece_id=VKWDPDYI4HJOU5UFLM2IYJNRGUB3M4TTY7B6V7ULA6C4N5ILKMNQ, seg_pos={0 0}, upload=55830]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 188.166.225.170:37318->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:32.069248118 (58h10m41.791811158s) [seg_pos={0 0}, upload=55830]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 26 21:46:32.296040812 (58h10m42.018603842s) [upload=55830]: Done uploading segments... Jul 26 21:46:32.296066651 (58h10m42.018629681s) [upload=55830]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Jul 26 21:46:37.441980998 (58h10m47.164544038s) [upload=55838]: Starting upload Jul 26 21:46:37.442040381 (58h10m47.164603421s) [upload=55838]: Done starting upload Jul 26 21:46:37.442233491 (58h10m47.164796531s) [upload=55838]: Uploading segments... Jul 26 21:46:37.442267826 (58h10m47.164830866s) [upload=55838]: Got next segment. Inline: false Jul 26 21:46:37.442289317 (58h10m47.164852358s) [upload=55838]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:46:37.442293355 (58h10m47.164856395s) [upload=55838]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:46:37.678545258 (58h10m47.401108298s) [seg_pos={0 0}, upload=55838]: Begin upload segment... Jul 26 21:46:37.678608138 (58h10m47.401171178s) [seg_pos={0 0}, upload=55838]: Done begin upload segment. Jul 26 21:46:37.678639779 (58h10m47.401202819s) [noise=true , address=104.36.82.5:4839 , piece_id=QDY4HHBJEJNM4ZA6QOH7RIUMTT6PDDWPHP3SDVTTBB5VJHXQCTYA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.678660478 (58h10m47.401223508s) [noise=true , address=104.36.82.16:4294 , piece_id=R6XZZVMF7SOY7AKR67A3GX3DFKYANJ2CKXC4NZJ7YEAAHQUB2ROA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.679009867 (58h10m47.401572907s) [noise=true , address=104.36.82.14:4223 , piece_id=VLT7N6DTNDU2Q7SSKUYFYLJQ2S5ESU5TYUF7SMHG2NPBZKDVRWBQ, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.679257030 (58h10m47.40182006s) [noise=true , address=45.91.116.10:4082 , piece_id=2HIQDNMMNEMH6SEPHUWZFPOQ6P76K32G7HBLB6NRMKTT5XZ6PG3A, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.679493953 (58h10m47.402056993s) [noise=true , address=45.91.116.14:4191 , piece_id=R5SR5FIICUTJ3VF2MORLOWSEMB35JGCWY4CIXM3BELOMSJLKYL2A, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.679879230 (58h10m47.40244226s) [noise=true , address=45.91.116.6:4975 , piece_id=WMZUAPGT5DL3MVMMN77OKY5CO5VJ6UKRVCDKOCEO7KAGWCWB4WLQ, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.680085715 (58h10m47.402648755s) [noise=true , address=104.36.82.14:4222 , piece_id=IIT4NC4WCNU3HAKHETAQSP6SUNVVXMTVIRW7AEIEN3X5QWL7V6AQ, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.680324222 (58h10m47.402887262s) [noise=true , address=45.91.116.7:4011 , piece_id=4YCQPHEMESPXCM7IBT4CK7BQEPBNDDLRU2CI2SP4J3DL3IAFLRUA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.680537320 (58h10m47.40310036s) [noise=true , address=45.91.116.11:4119 , piece_id=ZDJKBNAF44Q4HJLXVWHOXVTC3K4E42SUJD5L3XDZFB5TOXJPA4LQ, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.680719317 (58h10m47.403282347s) [noise=true , address=77.90.31.16:4734 , piece_id=6QRX3QBINYTPY2OK7VYYRCDAJC73EFFQR6QMEMSBIGTXFCLMCEJA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.680910824 (58h10m47.403473864s) [noise=true , address=45.91.116.4:4938 , piece_id=4WRIBTDPZ6BHEITZLSO5LBLPECWUNDXXKQBZGMI5TYHCFAJSUAWQ, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.681090358 (58h10m47.403653398s) [noise=true , address=45.91.116.12:4226 , piece_id=LOJCT3XSSLUSYSLVGIPV3KFSBDEHQ57AOLIXCAIBFODT5B3G3OTA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.681274990 (58h10m47.40383802s) [noise=true , address=45.91.116.9:4263 , piece_id=HRLBBRU5X5OT2CPJYKPB3CGMKHPVASMIVU72HL5DZUUSI4GZ3BJA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.681494711 (58h10m47.404057741s) [noise=true , address=45.91.116.14:4189 , piece_id=PH7PHBR5R3K32AEHSVR5GOBOKCIM45UPQB2TTQVLK7OFRHLD2RNA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.681622996 (58h10m47.404186036s) [noise=true , address=45.91.116.8:4048 , piece_id=D5MJRQHDJ6AHWVPS2GEW4XOSZXIRX2SV4N5E3HT5NRUG55BXWNGA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.681998685 (58h10m47.404561716s) [noise=true , address=77.90.31.10:4517 , piece_id=QRZEQP6GQVVEPOH2I6OXBYXYJYERI42ASR4W2PDYZGVZB6SA4Z7Q, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:39.412352944 (58h10m49.134915984s) [upload=55838]: Next returned nil segment Jul 26 21:46:39.412372020 (58h10m49.13493506s) [upload=55838]: Waiting for error group managing segments... Jul 26 21:46:39.688672748 (58h10m49.411235788s) [noise=true , address=45.91.116.11:4119 , piece_id=ZDJKBNAF44Q4HJLXVWHOXVTC3K4E42SUJD5L3XDZFB5TOXJPA4LQ, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.777267577 (58h10m49.499830617s) [noise=true , address=45.91.116.9:4263 , piece_id=HRLBBRU5X5OT2CPJYKPB3CGMKHPVASMIVU72HL5DZUUSI4GZ3BJA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.781489044 (58h10m49.504052084s) [noise=true , address=45.91.116.4:4938 , piece_id=4WRIBTDPZ6BHEITZLSO5LBLPECWUNDXXKQBZGMI5TYHCFAJSUAWQ, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.781588564 (58h10m49.504151604s) [noise=true , address=45.91.116.10:4082 , piece_id=2HIQDNMMNEMH6SEPHUWZFPOQ6P76K32G7HBLB6NRMKTT5XZ6PG3A, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.786455396 (58h10m49.509018436s) [noise=true , address=45.91.116.7:4011 , piece_id=4YCQPHEMESPXCM7IBT4CK7BQEPBNDDLRU2CI2SP4J3DL3IAFLRUA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.786893875 (58h10m49.509456915s) [noise=true , address=45.91.116.12:4226 , piece_id=LOJCT3XSSLUSYSLVGIPV3KFSBDEHQ57AOLIXCAIBFODT5B3G3OTA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.798085057 (58h10m49.520648097s) [noise=true , address=45.91.116.14:4189 , piece_id=PH7PHBR5R3K32AEHSVR5GOBOKCIM45UPQB2TTQVLK7OFRHLD2RNA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.857416065 (58h10m49.579979095s) [noise=true , address=45.91.116.8:4048 , piece_id=D5MJRQHDJ6AHWVPS2GEW4XOSZXIRX2SV4N5E3HT5NRUG55BXWNGA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.874964861 (58h10m49.597527921s) [noise=true , address=45.91.116.6:4975 , piece_id=WMZUAPGT5DL3MVMMN77OKY5CO5VJ6UKRVCDKOCEO7KAGWCWB4WLQ, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:40.011139835 (58h10m49.733702875s) [noise=true , address=45.91.116.14:4191 , piece_id=R5SR5FIICUTJ3VF2MORLOWSEMB35JGCWY4CIXM3BELOMSJLKYL2A, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:53.827404641 (58h11m3.549967681s) [noise=true , address=77.90.31.10:4517 , piece_id=QRZEQP6GQVVEPOH2I6OXBYXYJYERI42ASR4W2PDYZGVZB6SA4Z7Q, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:47:05.153186980 (58h11m14.87575002s) [noise=true , address=77.90.31.16:4734 , piece_id=6QRX3QBINYTPY2OK7VYYRCDAJC73EFFQR6QMEMSBIGTXFCLMCEJA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:47:06.793264488 (58h11m16.515827528s) [noise=true , address=104.36.82.14:4223 , piece_id=VLT7N6DTNDU2Q7SSKUYFYLJQ2S5ESU5TYUF7SMHG2NPBZKDVRWBQ, seg_pos={0 0}, upload=55838]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:47:06.793321647 (58h11m16.515884687s) [noise=true , address=104.36.82.16:4294 , piece_id=R6XZZVMF7SOY7AKR67A3GX3DFKYANJ2CKXC4NZJ7YEAAHQUB2ROA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:47:06.793500129 (58h11m16.516063169s) [noise=true , address=104.36.82.14:4222 , piece_id=IIT4NC4WCNU3HAKHETAQSP6SUNVVXMTVIRW7AEIEN3X5QWL7V6AQ, seg_pos={0 0}, upload=55838]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:47:06.793522010 (58h11m16.51608506s) [noise=true , address=104.36.82.5:4839 , piece_id=QDY4HHBJEJNM4ZA6QOH7RIUMTT6PDDWPHP3SDVTTBB5VJHXQCTYA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:104.36.82.5:4839): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:47:06.793540716 (58h11m16.516103756s) [seg_pos={0 0}, upload=55838]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:47:07.023763964 (58h11m16.746327004s) [upload=55838]: Done uploading segments... Jul 26 21:47:07.023790425 (58h11m16.746353465s) [upload=55838]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 14.186588514s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.....................................X....X......XX....XX.XX] duration: 30.231899337s FAIL (2025-07-26 21:47:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== C4WXBJRYY33IBBO6G2PKD6UHQYE73D7BEJA, seg_pos={0 0}, upload=55830]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:104.36.82.7:4912): protocol: rpc: dial tcp 104.36.82.7:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:32.069210546 (58h10m41.791773586s) [noise=true , address=104.36.82.4:4802 , piece_id=VKWDPDYI4HJOU5UFLM2IYJNRGUB3M4TTY7B6V7ULA6C4N5ILKMNQ, seg_pos={0 0}, upload=55830]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 188.166.225.170:37318->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:32.069248118 (58h10m41.791811158s) [seg_pos={0 0}, upload=55830]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 26 21:46:32.296040812 (58h10m42.018603842s) [upload=55830]: Done uploading segments... Jul 26 21:46:32.296066651 (58h10m42.018629681s) [upload=55830]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Jul 26 21:46:37.441980998 (58h10m47.164544038s) [upload=55838]: Starting upload Jul 26 21:46:37.442040381 (58h10m47.164603421s) [upload=55838]: Done starting upload Jul 26 21:46:37.442233491 (58h10m47.164796531s) [upload=55838]: Uploading segments... Jul 26 21:46:37.442267826 (58h10m47.164830866s) [upload=55838]: Got next segment. Inline: false Jul 26 21:46:37.442289317 (58h10m47.164852358s) [upload=55838]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:46:37.442293355 (58h10m47.164856395s) [upload=55838]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:46:37.678545258 (58h10m47.401108298s) [seg_pos={0 0}, upload=55838]: Begin upload segment... Jul 26 21:46:37.678608138 (58h10m47.401171178s) [seg_pos={0 0}, upload=55838]: Done begin upload segment. Jul 26 21:46:37.678639779 (58h10m47.401202819s) [noise=true , address=104.36.82.5:4839 , piece_id=QDY4HHBJEJNM4ZA6QOH7RIUMTT6PDDWPHP3SDVTTBB5VJHXQCTYA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.678660478 (58h10m47.401223508s) [noise=true , address=104.36.82.16:4294 , piece_id=R6XZZVMF7SOY7AKR67A3GX3DFKYANJ2CKXC4NZJ7YEAAHQUB2ROA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.679009867 (58h10m47.401572907s) [noise=true , address=104.36.82.14:4223 , piece_id=VLT7N6DTNDU2Q7SSKUYFYLJQ2S5ESU5TYUF7SMHG2NPBZKDVRWBQ, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.679257030 (58h10m47.40182006s) [noise=true , address=45.91.116.10:4082 , piece_id=2HIQDNMMNEMH6SEPHUWZFPOQ6P76K32G7HBLB6NRMKTT5XZ6PG3A, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.679493953 (58h10m47.402056993s) [noise=true , address=45.91.116.14:4191 , piece_id=R5SR5FIICUTJ3VF2MORLOWSEMB35JGCWY4CIXM3BELOMSJLKYL2A, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.679879230 (58h10m47.40244226s) [noise=true , address=45.91.116.6:4975 , piece_id=WMZUAPGT5DL3MVMMN77OKY5CO5VJ6UKRVCDKOCEO7KAGWCWB4WLQ, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.680085715 (58h10m47.402648755s) [noise=true , address=104.36.82.14:4222 , piece_id=IIT4NC4WCNU3HAKHETAQSP6SUNVVXMTVIRW7AEIEN3X5QWL7V6AQ, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.680324222 (58h10m47.402887262s) [noise=true , address=45.91.116.7:4011 , piece_id=4YCQPHEMESPXCM7IBT4CK7BQEPBNDDLRU2CI2SP4J3DL3IAFLRUA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.680537320 (58h10m47.40310036s) [noise=true , address=45.91.116.11:4119 , piece_id=ZDJKBNAF44Q4HJLXVWHOXVTC3K4E42SUJD5L3XDZFB5TOXJPA4LQ, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.680719317 (58h10m47.403282347s) [noise=true , address=77.90.31.16:4734 , piece_id=6QRX3QBINYTPY2OK7VYYRCDAJC73EFFQR6QMEMSBIGTXFCLMCEJA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.680910824 (58h10m47.403473864s) [noise=true , address=45.91.116.4:4938 , piece_id=4WRIBTDPZ6BHEITZLSO5LBLPECWUNDXXKQBZGMI5TYHCFAJSUAWQ, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.681090358 (58h10m47.403653398s) [noise=true , address=45.91.116.12:4226 , piece_id=LOJCT3XSSLUSYSLVGIPV3KFSBDEHQ57AOLIXCAIBFODT5B3G3OTA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.681274990 (58h10m47.40383802s) [noise=true , address=45.91.116.9:4263 , piece_id=HRLBBRU5X5OT2CPJYKPB3CGMKHPVASMIVU72HL5DZUUSI4GZ3BJA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.681494711 (58h10m47.404057741s) [noise=true , address=45.91.116.14:4189 , piece_id=PH7PHBR5R3K32AEHSVR5GOBOKCIM45UPQB2TTQVLK7OFRHLD2RNA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.681622996 (58h10m47.404186036s) [noise=true , address=45.91.116.8:4048 , piece_id=D5MJRQHDJ6AHWVPS2GEW4XOSZXIRX2SV4N5E3HT5NRUG55BXWNGA, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:37.681998685 (58h10m47.404561716s) [noise=true , address=77.90.31.10:4517 , piece_id=QRZEQP6GQVVEPOH2I6OXBYXYJYERI42ASR4W2PDYZGVZB6SA4Z7Q, seg_pos={0 0}, upload=55838]: Uploading piece... Jul 26 21:46:39.412352944 (58h10m49.134915984s) [upload=55838]: Next returned nil segment Jul 26 21:46:39.412372020 (58h10m49.13493506s) [upload=55838]: Waiting for error group managing segments... Jul 26 21:46:39.688672748 (58h10m49.411235788s) [noise=true , address=45.91.116.11:4119 , piece_id=ZDJKBNAF44Q4HJLXVWHOXVTC3K4E42SUJD5L3XDZFB5TOXJPA4LQ, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.777267577 (58h10m49.499830617s) [noise=true , address=45.91.116.9:4263 , piece_id=HRLBBRU5X5OT2CPJYKPB3CGMKHPVASMIVU72HL5DZUUSI4GZ3BJA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.781489044 (58h10m49.504052084s) [noise=true , address=45.91.116.4:4938 , piece_id=4WRIBTDPZ6BHEITZLSO5LBLPECWUNDXXKQBZGMI5TYHCFAJSUAWQ, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.781588564 (58h10m49.504151604s) [noise=true , address=45.91.116.10:4082 , piece_id=2HIQDNMMNEMH6SEPHUWZFPOQ6P76K32G7HBLB6NRMKTT5XZ6PG3A, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.786455396 (58h10m49.509018436s) [noise=true , address=45.91.116.7:4011 , piece_id=4YCQPHEMESPXCM7IBT4CK7BQEPBNDDLRU2CI2SP4J3DL3IAFLRUA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.786893875 (58h10m49.509456915s) [noise=true , address=45.91.116.12:4226 , piece_id=LOJCT3XSSLUSYSLVGIPV3KFSBDEHQ57AOLIXCAIBFODT5B3G3OTA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.798085057 (58h10m49.520648097s) [noise=true , address=45.91.116.14:4189 , piece_id=PH7PHBR5R3K32AEHSVR5GOBOKCIM45UPQB2TTQVLK7OFRHLD2RNA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.857416065 (58h10m49.579979095s) [noise=true , address=45.91.116.8:4048 , piece_id=D5MJRQHDJ6AHWVPS2GEW4XOSZXIRX2SV4N5E3HT5NRUG55BXWNGA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:39.874964861 (58h10m49.597527921s) [noise=true , address=45.91.116.6:4975 , piece_id=WMZUAPGT5DL3MVMMN77OKY5CO5VJ6UKRVCDKOCEO7KAGWCWB4WLQ, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:40.011139835 (58h10m49.733702875s) [noise=true , address=45.91.116.14:4191 , piece_id=R5SR5FIICUTJ3VF2MORLOWSEMB35JGCWY4CIXM3BELOMSJLKYL2A, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:46:53.827404641 (58h11m3.549967681s) [noise=true , address=77.90.31.10:4517 , piece_id=QRZEQP6GQVVEPOH2I6OXBYXYJYERI42ASR4W2PDYZGVZB6SA4Z7Q, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:47:05.153186980 (58h11m14.87575002s) [noise=true , address=77.90.31.16:4734 , piece_id=6QRX3QBINYTPY2OK7VYYRCDAJC73EFFQR6QMEMSBIGTXFCLMCEJA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: <nil> Jul 26 21:47:06.793264488 (58h11m16.515827528s) [noise=true , address=104.36.82.14:4223 , piece_id=VLT7N6DTNDU2Q7SSKUYFYLJQ2S5ESU5TYUF7SMHG2NPBZKDVRWBQ, seg_pos={0 0}, upload=55838]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:104.36.82.14:4223): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:47:06.793321647 (58h11m16.515884687s) [noise=true , address=104.36.82.16:4294 , piece_id=R6XZZVMF7SOY7AKR67A3GX3DFKYANJ2CKXC4NZJ7YEAAHQUB2ROA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:104.36.82.16:4294): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:47:06.793500129 (58h11m16.516063169s) [noise=true , address=104.36.82.14:4222 , piece_id=IIT4NC4WCNU3HAKHETAQSP6SUNVVXMTVIRW7AEIEN3X5QWL7V6AQ, seg_pos={0 0}, upload=55838]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:104.36.82.14:4222): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:47:06.793522010 (58h11m16.51608506s) [noise=true , address=104.36.82.5:4839 , piece_id=QDY4HHBJEJNM4ZA6QOH7RIUMTT6PDDWPHP3SDVTTBB5VJHXQCTYA, seg_pos={0 0}, upload=55838]: Done uploading piece. err: ecclient: upload failed (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv, address:104.36.82.5:4839): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:47:06.793540716 (58h11m16.516103756s) [seg_pos={0 0}, upload=55838]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:47:07.023763964 (58h11m16.746327004s) [upload=55838]: Done uploading segments... Jul 26 21:47:07.023790425 (58h11m16.746353465s) [upload=55838]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.231928303s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration: 1m0.239500179s FAIL (2025-07-26 21:46:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== piece_id=BVO2J4E4I7UUKI6PWBYB3BRXDS6QA76KH4MFR32JQL54QIJPRNJQ, seg_pos={0 0}, upload=55828]: Uploading piece... Jul 26 21:46:00.167489848 (58h10m9.890052888s) [noise=true , address=79.127.201.215:4360 , piece_id=QG5CPNOD26CODJPLIKTR4LRANAPSG6A4ZLZMGDW5T7JOHQQPWDEQ, seg_pos={0 0}, upload=55828]: Uploading piece... Jul 26 21:46:00.167840919 (58h10m9.890403969s) [noise=true , address=104.36.82.38:4710 , piece_id=AFK7OSGBACUYATBOCCSAGOAVUF6H6WTZLGVPYBZGNECXK77MHO7A, seg_pos={0 0}, upload=55828]: Uploading piece... Jul 26 21:46:00.168126396 (58h10m9.890689436s) [noise=true , address=104.36.82.47:4524 , piece_id=YE7TDOSA6Z4YG2U64OVNLPYWN7ASBBBUUGCHGQ5K6BHJIIMDQMJA, seg_pos={0 0}, upload=55828]: Uploading piece... Jul 26 21:46:00.168950142 (58h10m9.891513182s) [noise=true , address=104.36.82.55:4205 , piece_id=63ZUQQSRWVSPAVWHBLXXLK5QFT5IAKKSH3Y4HTQKRLH3ID3TK57Q, seg_pos={0 0}, upload=55828]: Uploading piece... Jul 26 21:46:00.169974432 (58h10m9.892537472s) [noise=true , address=64.49.13.140:4508 , piece_id=576MNPOPNEPPHOE5IBBPPYCGC6A3I4JEIKHQ4MDFVYK4HLCNKCBA, seg_pos={0 0}, upload=55828]: Uploading piece... Jul 26 21:46:00.170192629 (58h10m9.892755669s) [noise=true , address=79.127.205.235:4668 , piece_id=QNR7IHLDMXTC7MOEGQFMJUP6LQQ2P6TV445VV25PQNFPFD6ZM76A, seg_pos={0 0}, upload=55828]: Uploading piece... Jul 26 21:46:00.170426316 (58h10m9.892989356s) [noise=true , address=104.36.82.4:4820 , piece_id=LZFTDHLUYLJYHHL3A7HO6QVK2BLWVELRL42NP2KEFRO2BTMVLFYA, seg_pos={0 0}, upload=55828]: Uploading piece... Jul 26 21:46:00.171178345 (58h10m9.893741385s) [noise=true , address=104.36.82.17:4355 , piece_id=XOWNBTRQGI3Q2TBRNSDBAUG7KGZFPEPCFO4ILDVKREHCLL4YW4TA, seg_pos={0 0}, upload=55828]: Uploading piece... Jul 26 21:46:00.171466436 (58h10m9.894029476s) [noise=true , address=79.127.220.101:4048 , piece_id=GZ4I4VP3ZVABWJDQ74LFLJGEDOZDQYLAUM27KG7NAQRGSP5N5J7Q, seg_pos={0 0}, upload=55828]: Uploading piece... Jul 26 21:46:00.171747824 (58h10m9.894310864s) [noise=true , address=104.36.82.3:4769 , piece_id=LJCCRKOQMB3GO34FQ7TSBTEHQPOAXVLQJAGWBTZ3SAZK6F3XVAWQ, seg_pos={0 0}, upload=55828]: Uploading piece... Jul 26 21:46:00.171995718 (58h10m9.894558759s) [noise=true , address=104.36.82.56:4249 , piece_id=ME2RAYMNDJBYYD45KT6V53XCT5HP6SIHY3NWMY5IVULQL35VDJLQ, seg_pos={0 0}, upload=55828]: Uploading piece... Jul 26 21:46:00.172202064 (58h10m9.894765104s) [noise=true , address=104.36.82.7:4914 , piece_id=6VW4G6VB2HU55G6N3CUQD3U3CF7ZKHATIWX2PHGQZS5ZDNI3DXFA, seg_pos={0 0}, upload=55828]: Uploading piece... Jul 26 21:46:01.290631060 (58h10m11.0131941s) [upload=55828]: Next returned nil segment Jul 26 21:46:01.290648294 (58h10m11.013211334s) [upload=55828]: Waiting for error group managing segments... Jul 26 21:46:01.580053965 (58h10m11.302617005s) [noise=true , address=79.127.205.235:4668 , piece_id=QNR7IHLDMXTC7MOEGQFMJUP6LQQ2P6TV445VV25PQNFPFD6ZM76A, seg_pos={0 0}, upload=55828]: Done uploading piece. err: <nil> Jul 26 21:46:01.725193731 (58h10m11.447756772s) [noise=true , address=64.49.13.140:4508 , piece_id=576MNPOPNEPPHOE5IBBPPYCGC6A3I4JEIKHQ4MDFVYK4HLCNKCBA, seg_pos={0 0}, upload=55828]: Done uploading piece. err: <nil> Jul 26 21:46:02.050328204 (58h10m11.772891254s) [noise=true , address=79.127.220.101:4048 , piece_id=GZ4I4VP3ZVABWJDQ74LFLJGEDOZDQYLAUM27KG7NAQRGSP5N5J7Q, seg_pos={0 0}, upload=55828]: Done uploading piece. err: <nil> Jul 26 21:46:02.690912139 (58h10m12.413475179s) [noise=true , address=45.91.116.9:4269 , piece_id=P5L74SXRHXL75XJD7VCU4OYTR27XLLQWT4NKIMDO6QJS6WDKJUFA, seg_pos={0 0}, upload=55828]: Done uploading piece. err: <nil> Jul 26 21:46:03.405320368 (58h10m13.127883408s) [noise=true , address=79.127.201.215:4360 , piece_id=QG5CPNOD26CODJPLIKTR4LRANAPSG6A4ZLZMGDW5T7JOHQQPWDEQ, seg_pos={0 0}, upload=55828]: Done uploading piece. err: <nil> Jul 26 21:46:59.068135569 (58h11m8.790698609s) [noise=true , address=104.36.82.54:4178 , piece_id=YAGEV3JS2YGSDZLKV3IYQXNJUI6TWM6XPUEP2VUAXZGKNKODWX6Q, seg_pos={0 0}, upload=55828]: Done uploading piece. err: ecclient: upload failed (node:1jyRCtxSCu2bkU41Kgb1aeGppQLgX5iEo322Zoi1cWFkmGeYgr, address:104.36.82.54:4178): protocol: rpc: dial tcp 104.36.82.54:4178: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:59.068480410 (58h11m8.79104345s) [noise=true , address=104.36.82.52:4092 , piece_id=JU6BKFY423MEXCBWIYW7B4RWT3SAWMXIGBLNHZ3EMPQ6CDINGJPA, seg_pos={0 0}, upload=55828]: Done uploading piece. err: ecclient: upload failed (node:12wYaUu5QpFU8T4mZt1iqeHqVJKJ5CyVW5vkeTWf1a8G3pLsx3P, address:104.36.82.52:4092): protocol: rpc: dial tcp 104.36.82.52:4092: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:59.068532730 (58h11m8.79109577s) [noise=true , address=104.36.82.49:4989 , piece_id=7TTVRYNJ2QB2U7P6SVDCGLT7K6VHDA2X6L6QDJIBXDIV2SCXTD5A, seg_pos={0 0}, upload=55828]: Done uploading piece. err: ecclient: upload failed (node:1boeQC5mQx4MVWhWAJFs3g4DtJMpT1tw4QYrZQ3c2RbDjRAAJc, address:104.36.82.49:4989): protocol: rpc: dial tcp 104.36.82.49:4989: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:59.068568438 (58h11m8.791131468s) [noise=true , address=104.36.82.46:4495 , piece_id=BVO2J4E4I7UUKI6PWBYB3BRXDS6QA76KH4MFR32JQL54QIJPRNJQ, seg_pos={0 0}, upload=55828]: Done uploading piece. err: ecclient: upload failed (node:1YpHarCJ3ouMGTs59udhU7az9h68ELEHTa2aTAax2chR2Gux1x, address:104.36.82.46:4495): protocol: rpc: dial tcp 104.36.82.46:4495: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:59.068599437 (58h11m8.791162477s) [noise=true , address=104.36.82.53:4138 , piece_id=3J7CES3WQRTLCSGNS3YGFMVBTNANITSEYW4A6BVK46G2UHU4L6OQ, seg_pos={0 0}, upload=55828]: Done uploading piece. err: ecclient: upload failed (node:1YUGGJg6mWeQMmzSNWtLFWAaWwaKWfEf8wxXs1jwiZaA9nhsr3, address:104.36.82.53:4138): protocol: rpc: dial tcp 104.36.82.53:4138: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:59.068635096 (58h11m8.791198126s) [noise=true , address=104.36.82.4:4820 , piece_id=LZFTDHLUYLJYHHL3A7HO6QVK2BLWVELRL42NP2KEFRO2BTMVLFYA, seg_pos={0 0}, upload=55828]: Done uploading piece. err: ecclient: upload failed (node:12TnUNGbwHSf335XW7vNuq1ffwhFkgxomckBQmF189779tRU59k, address:104.36.82.4:4820): protocol: write tcp 188.166.225.170:47404->104.36.82.4:4820: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:59.068662318 (58h11m8.791225358s) [noise=true , address=104.36.82.38:4710 , piece_id=AFK7OSGBACUYATBOCCSAGOAVUF6H6WTZLGVPYBZGNECXK77MHO7A, seg_pos={0 0}, upload=55828]: Done uploading piece. err: ecclient: upload failed (node:12JPDQdVWuu5QDXYMAj6MWpaeuKQxdzBdo2t5R7Z25NYXgouscM, address:104.36.82.38:4710): protocol: rpc: dial tcp 104.36.82.38:4710: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:59.068715941 (58h11m8.791278981s) [noise=true , address=104.36.82.47:4524 , piece_id=YE7TDOSA6Z4YG2U64OVNLPYWN7ASBBBUUGCHGQ5K6BHJIIMDQMJA, seg_pos={0 0}, upload=55828]: Done uploading piece. err: ecclient: upload failed (node:1xfbCzjSwFDRrR1gPggeN8tNCLbRRWDVzCJ6mqhZsbVSe8d4s4, address:104.36.82.47:4524): protocol: rpc: dial tcp 104.36.82.47:4524: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:59.068757319 (58h11m8.791320359s) [noise=true , address=104.36.82.3:4769 , piece_id=LJCCRKOQMB3GO34FQ7TSBTEHQPOAXVLQJAGWBTZ3SAZK6F3XVAWQ, seg_pos={0 0}, upload=55828]: Done uploading piece. err: ecclient: upload failed (node:1CNSBmPmCSQ6gHKNcWrFAy3VU7Rf2ukMYQpkdurgZLPmqotWQ6, address:104.36.82.3:4769): protocol: rpc: dial tcp 104.36.82.3:4769: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:59.068807075 (58h11m8.791370115s) [noise=true , address=104.36.82.56:4249 , piece_id=ME2RAYMNDJBYYD45KT6V53XCT5HP6SIHY3NWMY5IVULQL35VDJLQ, seg_pos={0 0}, upload=55828]: Done uploading piece. err: ecclient: upload failed (node:1LuzmiCefHA9kX3HHWHqviHfG4Ts2t5DYu5nfJRWquaQ8nQrzx, address:104.36.82.56:4249): protocol: rpc: dial tcp 104.36.82.56:4249: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:59.068931623 (58h11m8.791494663s) [noise=true , address=104.36.82.7:4914 , piece_id=6VW4G6VB2HU55G6N3CUQD3U3CF7ZKHATIWX2PHGQZS5ZDNI3DXFA, seg_pos={0 0}, upload=55828]: Done uploading piece. err: ecclient: upload failed (node:1MArPyMrdvyw4VFuArmattTtneNTaN5mt82nfGwY2sKHvQnL5D, address:104.36.82.7:4914): protocol: rpc: dial tcp 104.36.82.7:4914: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:59.068971409 (58h11m8.791534439s) [noise=true , address=104.36.82.55:4205 , piece_id=63ZUQQSRWVSPAVWHBLXXLK5QFT5IAKKSH3Y4HTQKRLH3ID3TK57Q, seg_pos={0 0}, upload=55828]: Done uploading piece. err: ecclient: upload failed (node:12UafxBVo9nuRVBcQmGE4gBP2pnXQ5YUrwHZdwPCeSBSBWCZKpQ, address:104.36.82.55:4205): protocol: rpc: dial tcp 104.36.82.55:4205: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:59.068991358 (58h11m8.791554398s) [noise=true , address=104.36.82.17:4355 , piece_id=XOWNBTRQGI3Q2TBRNSDBAUG7KGZFPEPCFO4ILDVKREHCLL4YW4TA, seg_pos={0 0}, upload=55828]: Done uploading piece. err: ecclient: upload failed (node:12mRuEpX7CMxWXVdHrTi3zF3v9t3LcqE6mJKzocajEvYHM52a7T, address:104.36.82.17:4355): protocol: rpc: dial tcp 104.36.82.17:4355: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:59.069034790 (58h11m8.79159783s) [seg_pos={0 0}, upload=55828]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 26 21:46:59.305968176 (58h11m9.028531216s) [upload=55828]: Done uploading segments... Jul 26 21:46:59.305995077 (58h11m9.028558117s) [upload=55828]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 1m0.239512132s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.......X...................................XX.........XX.XX.] duration: 26.533432135s FAIL (2025-07-26 21:46:47 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 26.533451493s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...............X.........X.................X................] duration: 1.841647443s FAIL (2025-07-26 21:46:43 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.841669946s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [......................................X....X......XX....XX.X] duration: 30.22852853s FAIL (2025-07-26 21:46:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== =55817]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:45:20.592715405 (58h9m30.315278445s) [upload=55817]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:45:20.892270414 (58h9m30.614833454s) [upload=55817]: Done uploading segments... Jul 26 21:45:20.892293949 (58h9m30.614856999s) [upload=55817]: Upload finished. err: <nil> Jul 26 21:46:02.710358585 (58h10m12.432921625s) [upload=55830]: Starting upload Jul 26 21:46:02.710414683 (58h10m12.432977723s) [upload=55830]: Done starting upload Jul 26 21:46:02.710468976 (58h10m12.433032016s) [upload=55830]: Uploading segments... Jul 26 21:46:02.710609986 (58h10m12.433173026s) [upload=55830]: Got next segment. Inline: false Jul 26 21:46:02.710633541 (58h10m12.433196581s) [upload=55830]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:46:02.710638080 (58h10m12.43320112s) [upload=55830]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:46:02.942466998 (58h10m12.665030038s) [seg_pos={0 0}, upload=55830]: Begin upload segment... Jul 26 21:46:02.942516922 (58h10m12.665079963s) [seg_pos={0 0}, upload=55830]: Done begin upload segment. Jul 26 21:46:02.942544706 (58h10m12.665107746s) [noise=true , address=104.36.82.16:4293 , piece_id=6D6KZADD6OKM7OCGRFQCYZCCRMAF5KVOCUBAPM723CDPDHXTNCPA, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.942551479 (58h10m12.665114519s) [noise=true , address=45.91.116.4:4940 , piece_id=Z5CZQ6UQEPJJG5S5MLTGIVFXO3THLUOMR44BQHOF5YTWHVZ5XI7A, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.942588470 (58h10m12.66515151s) [noise=true , address=77.90.31.11:4555 , piece_id=5VZZRIHV3CPBAHWKD6AVML7KHQUZXHIY4JHHNKFVBH7PNGVGD4FA, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.942878665 (58h10m12.665441716s) [noise=true , address=104.36.82.4:4802 , piece_id=VKWDPDYI4HJOU5UFLM2IYJNRGUB3M4TTY7B6V7ULA6C4N5ILKMNQ, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.942951334 (58h10m12.665514374s) [noise=true , address=45.91.116.6:4975 , piece_id=L3FWKRNZTH67TGAPD5FEFQ42ONL3VBZ7OCMNFZUOYIQW4245FY4Q, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.943116149 (58h10m12.665679189s) [noise=true , address=104.36.82.8:4972 , piece_id=3OYBKPCCMJJVEDGBRUU45PPERFOKDG5MBHFPD7QJC5HADT5OHP7Q, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.943342742 (58h10m12.665905783s) [noise=true , address=45.91.116.9:4263 , piece_id=OPO62QNET3FBWEF2QJ3JIZA4S6JWWHWF4NGCH4X3LGAIPM4G6ETQ, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.943488342 (58h10m12.666051382s) [noise=true , address=45.91.116.14:4191 , piece_id=WKLHW2OT5F6KHK2GL4M7KELJEW4WEQFAXZTVAOHA7DNEOQOJVMCA, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.943570609 (58h10m12.666133649s) [noise=true , address=77.90.31.19:4843 , piece_id=R2TXXI75PYFZ3R3W5YZBU5U34SVSX76NPLVHBKCCLLTPV4NTY2FQ, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.943783776 (58h10m12.666346816s) [noise=true , address=45.91.116.7:4010 , piece_id=DBON34M3K6VEDBUXVFUJNHO3T4BQBRME4EJTEEBELFAXSKIPGGIQ, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.944010621 (58h10m12.666573661s) [noise=true , address=104.36.82.7:4912 , piece_id=WD4LWDVI6ZVIV2B2BC4WXBJRYY33IBBO6G2PKD6UHQYE73D7BEJA, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.944218769 (58h10m12.666781809s) [noise=true , address=45.91.116.11:4117 , piece_id=M4E4NGCIHN3ZZO3QUCUVBWOU67T2GMEC745RYBV7HB6S5UB7SG5A, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.944492713 (58h10m12.667055753s) [noise=true , address=45.91.116.10:4081 , piece_id=SRKZIF77NPP4BQNL55KD5JQ7Y2NF7KHUHIOOREXCMLZ423LCX4HQ, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.944719778 (58h10m12.667282818s) [noise=true , address=45.91.116.12:4226 , piece_id=GTVYY3Z3U6ZP6GWO4Z2DSNX7KXUTFNW5YMTWGAAJYAXH3RMNHZWA, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.944967191 (58h10m12.667530232s) [noise=true , address=45.91.116.8:4048 , piece_id=W3C53KL4XYX5XDH7E636VK5QYFJUARU6ZPDWGKEBCILFUW5ZABOA, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:02.945163948 (58h10m12.667726988s) [noise=true , address=45.91.116.5:4299 , piece_id=RXXOYXNSW3ATUPAX2HOF4X2UNW3FF6XLNO75GFN656DVWLQRUMHA, seg_pos={0 0}, upload=55830]: Uploading piece... Jul 26 21:46:04.669708798 (58h10m14.392271828s) [upload=55830]: Next returned nil segment Jul 26 21:46:04.669755908 (58h10m14.392318948s) [upload=55830]: Waiting for error group managing segments... Jul 26 21:46:04.946160495 (58h10m14.668723535s) [noise=true , address=45.91.116.12:4226 , piece_id=GTVYY3Z3U6ZP6GWO4Z2DSNX7KXUTFNW5YMTWGAAJYAXH3RMNHZWA, seg_pos={0 0}, upload=55830]: Done uploading piece. err: <nil> Jul 26 21:46:04.956961981 (58h10m14.679525012s) [noise=true , address=45.91.116.14:4191 , piece_id=WKLHW2OT5F6KHK2GL4M7KELJEW4WEQFAXZTVAOHA7DNEOQOJVMCA, seg_pos={0 0}, upload=55830]: Done uploading piece. err: <nil> Jul 26 21:46:04.962849957 (58h10m14.685412997s) [noise=true , address=45.91.116.5:4299 , piece_id=RXXOYXNSW3ATUPAX2HOF4X2UNW3FF6XLNO75GFN656DVWLQRUMHA, seg_pos={0 0}, upload=55830]: Done uploading piece. err: <nil> Jul 26 21:46:05.040380917 (58h10m14.762943957s) [noise=true , address=45.91.116.8:4048 , piece_id=W3C53KL4XYX5XDH7E636VK5QYFJUARU6ZPDWGKEBCILFUW5ZABOA, seg_pos={0 0}, upload=55830]: Done uploading piece. err: <nil> Jul 26 21:46:05.044417020 (58h10m14.76698006s) [noise=true , address=45.91.116.9:4263 , piece_id=OPO62QNET3FBWEF2QJ3JIZA4S6JWWHWF4NGCH4X3LGAIPM4G6ETQ, seg_pos={0 0}, upload=55830]: Done uploading piece. err: <nil> Jul 26 21:46:05.060235275 (58h10m14.782798315s) [noise=true , address=45.91.116.4:4940 , piece_id=Z5CZQ6UQEPJJG5S5MLTGIVFXO3THLUOMR44BQHOF5YTWHVZ5XI7A, seg_pos={0 0}, upload=55830]: Done uploading piece. err: <nil> Jul 26 21:46:05.114710142 (58h10m14.837273182s) [noise=true , address=45.91.116.10:4081 , piece_id=SRKZIF77NPP4BQNL55KD5JQ7Y2NF7KHUHIOOREXCMLZ423LCX4HQ, seg_pos={0 0}, upload=55830]: Done uploading piece. err: <nil> Jul 26 21:46:05.119547327 (58h10m14.842110367s) [noise=true , address=45.91.116.6:4975 , piece_id=L3FWKRNZTH67TGAPD5FEFQ42ONL3VBZ7OCMNFZUOYIQW4245FY4Q, seg_pos={0 0}, upload=55830]: Done uploading piece. err: <nil> Jul 26 21:46:05.138847033 (58h10m14.861410073s) [noise=true , address=45.91.116.11:4117 , piece_id=M4E4NGCIHN3ZZO3QUCUVBWOU67T2GMEC745RYBV7HB6S5UB7SG5A, seg_pos={0 0}, upload=55830]: Done uploading piece. err: <nil> Jul 26 21:46:05.192045939 (58h10m14.914608969s) [noise=true , address=45.91.116.7:4010 , piece_id=DBON34M3K6VEDBUXVFUJNHO3T4BQBRME4EJTEEBELFAXSKIPGGIQ, seg_pos={0 0}, upload=55830]: Done uploading piece. err: <nil> Jul 26 21:46:32.068873882 (58h10m41.791436922s) [noise=true , address=77.90.31.19:4843 , piece_id=R2TXXI75PYFZ3R3W5YZBU5U34SVSX76NPLVHBKCCLLTPV4NTY2FQ, seg_pos={0 0}, upload=55830]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: expected piece hash; context deadline exceeded; EOF Jul 26 21:46:32.068932444 (58h10m41.791495484s) [noise=true , address=104.36.82.16:4293 , piece_id=6D6KZADD6OKM7OCGRFQCYZCCRMAF5KVOCUBAPM723CDPDHXTNCPA, seg_pos={0 0}, upload=55830]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:104.36.82.16:4293): protocol: rpc: dial tcp 104.36.82.16:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:32.069010954 (58h10m41.791573994s) [noise=true , address=104.36.82.8:4972 , piece_id=3OYBKPCCMJJVEDGBRUU45PPERFOKDG5MBHFPD7QJC5HADT5OHP7Q, seg_pos={0 0}, upload=55830]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:104.36.82.8:4972): protocol: rpc: dial tcp 104.36.82.8:4972: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:32.069172343 (58h10m41.791735383s) [noise=true , address=77.90.31.11:4555 , piece_id=5VZZRIHV3CPBAHWKD6AVML7KHQUZXHIY4JHHNKFVBH7PNGVGD4FA, seg_pos={0 0}, upload=55830]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: write tcp 188.166.225.170:38536->77.90.31.11:4555: use of closed network connection; write tcp 188.166.225.170:38536->77.90.31.11:4555: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38536->77.90.31.11:4555: use of closed network connection Jul 26 21:46:32.069194546 (58h10m41.791757586s) [noise=true , address=104.36.82.7:4912 , piece_id=WD4LWDVI6ZVIV2B2BC4WXBJRYY33IBBO6G2PKD6UHQYE73D7BEJA, seg_pos={0 0}, upload=55830]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:104.36.82.7:4912): protocol: rpc: dial tcp 104.36.82.7:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:32.069210546 (58h10m41.791773586s) [noise=true , address=104.36.82.4:4802 , piece_id=VKWDPDYI4HJOU5UFLM2IYJNRGUB3M4TTY7B6V7ULA6C4N5ILKMNQ, seg_pos={0 0}, upload=55830]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:104.36.82.4:4802): protocol: write tcp 188.166.225.170:37318->104.36.82.4:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:46:32.069248118 (58h10m41.791811158s) [seg_pos={0 0}, upload=55830]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Jul 26 21:46:32.296040812 (58h10m42.018603842s) [upload=55830]: Done uploading segments... Jul 26 21:46:32.296066651 (58h10m42.018629681s) [upload=55830]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.22854972s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [........X...................................XX.........XX.XX] duration: 30.231947276s FAIL (2025-07-26 21:46:20 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.231994516s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [................X.........X.................X...............] duration: 1.869298379s FAIL (2025-07-26 21:46:03 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.869325711s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 17 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXXXXXXX] duration: 1m0.234425427s FAIL (2025-07-26 21:45:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 7) ======LOGS====== 5RRTHPJFS5IYKDVRLKCQLDJLFCR3VKL5EHA, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:44:59.789271069 (58h9m9.511834109s) [noise=true , address=104.36.82.14:4252 , piece_id=VCZN4XHX7MYMNXDWQMVNYZWLLLO5UCWPA4VNYEVXSA767V2ML3QQ, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:44:59.789474940 (58h9m9.51203798s) [noise=true , address=104.36.82.53:4136 , piece_id=SPQ2LFST3WB3GZE6BT2TTWNMRTUSGSS3XITRCE73Y6S7AUJZPJMA, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:44:59.789787517 (58h9m9.512350547s) [noise=true , address=104.36.82.55:4201 , piece_id=A4LWSN32KM3BCFREC6AIFEFQPWSMJRWQ46D5WZYMF6742E7KPWKA, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:44:59.790256555 (58h9m9.512819595s) [noise=true , address=79.127.219.38:4001 , piece_id=5WJM7MET7PTWUYVYK4M2TKME2ESADXTDLZ7XMLHB3D7E5PLOPBVQ, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:44:59.790609580 (58h9m9.51317262s) [noise=true , address=104.36.82.54:4175 , piece_id=EUTK2WHQIEMEWMHQNECRV5SOF75VRI36BRIIM6QN5STQKWMYQJPQ, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:44:59.791078728 (58h9m9.513641768s) [noise=true , address=104.36.82.56:4234 , piece_id=EKXDIE7JDBHDQ7O6ZXB37EG4JLN7EDI4MBEOPIBL7CVRDVW2IA3A, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:44:59.791428768 (58h9m9.513991808s) [noise=true , address=104.36.82.36:4624 , piece_id=EG5KOP54YXOKCEPZUBCXKBVTR7PGUX6SBQHHY2F4OKP2GVOPRDVA, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:44:59.791672294 (58h9m9.514235324s) [noise=true , address=104.36.82.6:4879 , piece_id=GAXSOHQSUSVBEFSSCIIRPIYR3LAO6V2DGYMBZA6YWE2CW35HDCYA, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:44:59.791950155 (58h9m9.514513186s) [noise=true , address=79.127.205.246:4282 , piece_id=VZTCWIRTH3NIIJW5SHT2CFFJNQR4MRTDHSNDICKKV3VESYTQNDBA, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:44:59.792222607 (58h9m9.514785657s) [noise=true , address=104.36.82.50:4046 , piece_id=BLFSDEPM52MIO4ZFONVYLZHBCKRY6JWQJANJJELV6U2GUMKWH52A, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:44:59.792638603 (58h9m9.515201643s) [noise=true , address=64.49.13.155:4916 , piece_id=UV46GAWN6ZOFEC5GQ2Z54BAWH65AMDSDQLTQDSAJT35K7IYIO3ZA, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:44:59.793026234 (58h9m9.515589274s) [noise=true , address=79.127.201.215:4354 , piece_id=J33OQIBCH2SAGO5MWZBKOS34MZBXAN2QKORXPQMQXIL6FFL6C5TA, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:44:59.793393196 (58h9m9.515956226s) [noise=true , address=104.36.82.46:4509 , piece_id=4BYNUQTPFQMHFIKWSIK4MU42N2IJW6CUJ5V36NGYX3O4UOV52U3A, seg_pos={0 0}, upload=55812]: Uploading piece... Jul 26 21:45:00.894815593 (58h9m10.617378633s) [upload=55812]: Next returned nil segment Jul 26 21:45:00.894835672 (58h9m10.617398712s) [upload=55812]: Waiting for error group managing segments... Jul 26 21:45:01.191146244 (58h9m10.913709284s) [noise=true , address=79.127.205.233:4652 , piece_id=V4JMMRQCVCTNAYSXF5RRTHPJFS5IYKDVRLKCQLDJLFCR3VKL5EHA, seg_pos={0 0}, upload=55812]: Done uploading piece. err: <nil> Jul 26 21:45:01.204821169 (58h9m10.927384219s) [noise=true , address=79.127.205.246:4282 , piece_id=VZTCWIRTH3NIIJW5SHT2CFFJNQR4MRTDHSNDICKKV3VESYTQNDBA, seg_pos={0 0}, upload=55812]: Done uploading piece. err: <nil> Jul 26 21:45:01.253875405 (58h9m10.976438455s) [noise=true , address=79.127.205.245:4236 , piece_id=BYNEZZICETXHX54YC5DNGCCERVTMF2WT5WKFJA6BHTFG7HFQGKPA, seg_pos={0 0}, upload=55812]: Done uploading piece. err: <nil> Jul 26 21:45:01.693649217 (58h9m11.416212267s) [noise=true , address=79.127.201.215:4354 , piece_id=J33OQIBCH2SAGO5MWZBKOS34MZBXAN2QKORXPQMQXIL6FFL6C5TA, seg_pos={0 0}, upload=55812]: Done uploading piece. err: <nil> Jul 26 21:45:02.242961009 (58h9m11.96552405s) [noise=true , address=64.49.13.155:4916 , piece_id=UV46GAWN6ZOFEC5GQ2Z54BAWH65AMDSDQLTQDSAJT35K7IYIO3ZA, seg_pos={0 0}, upload=55812]: Done uploading piece. err: <nil> Jul 26 21:45:07.683317775 (58h9m17.405880815s) [noise=true , address=79.127.219.38:4001 , piece_id=5WJM7MET7PTWUYVYK4M2TKME2ESADXTDLZ7XMLHB3D7E5PLOPBVQ, seg_pos={0 0}, upload=55812]: Done uploading piece. err: <nil> Jul 26 21:45:19.363152506 (58h9m29.085715546s) [noise=true , address=77.90.31.11:4576 , piece_id=7LQ5LZSERVKDFG2GY7RWN5W2AGEE4BDRQQ4PF64WSTVD6AQ7AD5A, seg_pos={0 0}, upload=55812]: Done uploading piece. err: <nil> Jul 26 21:45:58.605358634 (58h10m8.327921664s) [noise=true , address=104.36.82.47:4521 , piece_id=LOVWOAZTCYBW6PVWWVZHII754WPO6BEZOQIKZ54UKHWFMYE3WDWQ, seg_pos={0 0}, upload=55812]: Done uploading piece. err: ecclient: upload failed (node:12GQHySYUuqERoQyP4fCib2nRMNJhgpP2NkDzVxz6nurLkeJ5VD, address:104.36.82.47:4521): protocol: write tcp 188.166.225.170:49178->104.36.82.47:4521: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:45:58.605407027 (58h10m8.327970067s) [noise=true , address=104.36.82.36:4624 , piece_id=EG5KOP54YXOKCEPZUBCXKBVTR7PGUX6SBQHHY2F4OKP2GVOPRDVA, seg_pos={0 0}, upload=55812]: Done uploading piece. err: ecclient: upload failed (node:1GG1c98sVuVmeJgmwXuVqxm7hKZ7Wv3LgwQGzgzGtYETe4RbF1, address:104.36.82.36:4624): protocol: write tcp 188.166.225.170:33428->104.36.82.36:4624: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:45:58.605415613 (58h10m8.327978653s) [noise=true , address=104.36.82.55:4201 , piece_id=A4LWSN32KM3BCFREC6AIFEFQPWSMJRWQ46D5WZYMF6742E7KPWKA, seg_pos={0 0}, upload=55812]: Done uploading piece. err: ecclient: upload failed (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi, address:104.36.82.55:4201): protocol: rpc: dial tcp 104.36.82.55:4201: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:45:58.605443126 (58h10m8.328006166s) [noise=true , address=104.36.82.46:4509 , piece_id=4BYNUQTPFQMHFIKWSIK4MU42N2IJW6CUJ5V36NGYX3O4UOV52U3A, seg_pos={0 0}, upload=55812]: Done uploading piece. err: ecclient: upload failed (node:1y2NmauNvCyrMJ5z66Lihb3h3C9nwdJYqfSfNmC5dwDGYQqUKh, address:104.36.82.46:4509): protocol: write tcp 188.166.225.170:48446->104.36.82.46:4509: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:45:58.605447614 (58h10m8.328010654s) [noise=true , address=104.36.82.14:4252 , piece_id=VCZN4XHX7MYMNXDWQMVNYZWLLLO5UCWPA4VNYEVXSA767V2ML3QQ, seg_pos={0 0}, upload=55812]: Done uploading piece. err: ecclient: upload failed (node:125gP7pSQx5QXXgE7boHtFAtBqaZ19Ti144TEhWdJYwxCzncvCs, address:104.36.82.14:4252): protocol: rpc: dial tcp 104.36.82.14:4252: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:45:58.605472702 (58h10m8.328035742s) [noise=true , address=104.36.82.53:4136 , piece_id=SPQ2LFST3WB3GZE6BT2TTWNMRTUSGSS3XITRCE73Y6S7AUJZPJMA, seg_pos={0 0}, upload=55812]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:104.36.82.53:4136): protocol: write tcp 188.166.225.170:54274->104.36.82.53:4136: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:45:58.605478734 (58h10m8.328041774s) [noise=true , address=104.36.82.54:4175 , piece_id=EUTK2WHQIEMEWMHQNECRV5SOF75VRI36BRIIM6QN5STQKWMYQJPQ, seg_pos={0 0}, upload=55812]: Done uploading piece. err: ecclient: upload failed (node:12ou7jwFyJP3mNoALddniUTDT9duVyQKZjeq4xHCyx6sRWpwEPJ, address:104.36.82.54:4175): protocol: rpc: dial tcp 104.36.82.54:4175: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:45:58.605500626 (58h10m8.328063666s) [noise=true , address=104.36.82.6:4879 , piece_id=GAXSOHQSUSVBEFSSCIIRPIYR3LAO6V2DGYMBZA6YWE2CW35HDCYA, seg_pos={0 0}, upload=55812]: Done uploading piece. err: ecclient: upload failed (node:12jH7W99nQ77T67b8WVVqmg9Jqi5QoB1qM5Wcj8WexywAZ7fRNT, address:104.36.82.6:4879): protocol: write tcp 188.166.225.170:59896->104.36.82.6:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:45:58.605526315 (58h10m8.328089355s) [noise=true , address=104.36.82.50:4046 , piece_id=BLFSDEPM52MIO4ZFONVYLZHBCKRY6JWQJANJJELV6U2GUMKWH52A, seg_pos={0 0}, upload=55812]: Done uploading piece. err: ecclient: upload failed (node:12EsoehnMdbRs4cRP66HD93ZWZuzpMWYkfo1VxvQLyQcodAEm7C, address:104.36.82.50:4046): protocol: write tcp 188.166.225.170:52696->104.36.82.50:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:45:58.605533989 (58h10m8.328097029s) [noise=true , address=104.36.82.10:4100 , piece_id=RTJLPWWF4KUY47GWQS57MAMBJLA4QE6SYUYUVCZGR7PC6NJK2N6A, seg_pos={0 0}, upload=55812]: Done uploading piece. err: ecclient: upload failed (node:12Hyc6sbtuPXMLyFzubMcepbJyLyVAGNpkXAmqg73i3sY97dwdW, address:104.36.82.10:4100): protocol: write tcp 188.166.225.170:53260->104.36.82.10:4100: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:45:58.605549760 (58h10m8.3281128s) [noise=true , address=104.36.82.56:4234 , piece_id=EKXDIE7JDBHDQ7O6ZXB37EG4JLN7EDI4MBEOPIBL7CVRDVW2IA3A, seg_pos={0 0}, upload=55812]: Done uploading piece. err: ecclient: upload failed (node:12uthP31WaBmn37h54Q6z9ZMtYRY6K8917Ms4kwyyADqu4EXPJy, address:104.36.82.56:4234): protocol: write tcp 188.166.225.170:56810->104.36.82.56:4234: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:45:58.605568606 (58h10m8.328131646s) [seg_pos={0 0}, upload=55812]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil> Jul 26 21:45:58.837695335 (58h10m8.560258415s) [upload=55812]: Done uploading segments... Jul 26 21:45:58.837728258 (58h10m8.560291348s) [upload=55812]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7) Duration: 1m0.234444154s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........X...................................XX.........XX.X] duration: 30.235901322s FAIL (2025-07-26 21:45:38 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.235928714s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.................X.........X.................X..............] duration: 1.817042127s FAIL (2025-07-26 21:45:32 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.817071814s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......................................X....X......XX....XX.] duration: 14.134778368s FAIL (2025-07-26 21:45:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 15897 (58h8m17.709978927s) [upload=55790]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:44:07.987421177 (58h8m17.709984217s) [upload=55790]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:44:08.282987134 (58h8m18.005550174s) [upload=55790]: Done uploading segments... Jul 26 21:44:08.283006010 (58h8m18.005569051s) [upload=55790]: Upload finished. err: <nil> Jul 26 21:44:19.861345498 (58h8m29.583908538s) [upload=55800]: Starting upload Jul 26 21:44:19.861415974 (58h8m29.583979024s) [upload=55800]: Done starting upload Jul 26 21:44:19.861529861 (58h8m29.584092891s) [upload=55800]: Uploading segments... Jul 26 21:44:19.861610205 (58h8m29.584173235s) [upload=55800]: Got next segment. Inline: false Jul 26 21:44:19.861637628 (58h8m29.584200668s) [upload=55800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:44:19.861641836 (58h8m29.584204876s) [upload=55800]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:44:20.094447584 (58h8m29.817010624s) [seg_pos={0 0}, upload=55800]: Begin upload segment... Jul 26 21:44:20.094481459 (58h8m29.817044499s) [seg_pos={0 0}, upload=55800]: Done begin upload segment. Jul 26 21:44:20.094499433 (58h8m29.817062473s) [noise=true , address=45.91.116.9:4262 , piece_id=4ZRP7DXDAAB4AXW6G4CBDONFRETI4I4JO3APCR4PU6OEUK3VBXLQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.094532116 (58h8m29.817095146s) [noise=true , address=77.90.31.16:4734 , piece_id=EVKGL35AJ4PC2VCOIHSJVW4HE7FWT2X3X2RI3UJNWMPCHPX5K2BA, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.094778667 (58h8m29.817341707s) [noise=true , address=45.91.116.5:4298 , piece_id=K3M3OTOPARCSNS2SQMULYQIQHTMMXAERDTOJIMG56AQPCCU42GQA, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.094935467 (58h8m29.817498497s) [noise=true , address=104.36.82.7:4912 , piece_id=XZ6GFCGJBG457FEG2COTVV3HG7YKJFK66ZSIHUET2BZ5YOWDWVOQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.095034998 (58h8m29.817598038s) [noise=true , address=77.90.31.15:4697 , piece_id=42FVGSO2I7MS4BLUYQVP2XELP63BBADQ4RGZD6YJZ3RCUDQHIUTQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.095218238 (58h8m29.817781279s) [noise=true , address=45.91.116.10:4082 , piece_id=DVKDGWK6ZH6TPG34U24OWOQAJSCT5CXIWZST6M7V5CUOEN4WQHPA, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.095277131 (58h8m29.817840171s) [noise=true , address=77.90.31.21:4877 , piece_id=AAWMVDGF5YY43Q5CJBS6TGT7JRAGBRG5HFIQIZFZP2RJVI7QJCMQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.095473016 (58h8m29.818036056s) [noise=true , address=77.90.31.17:4771 , piece_id=IBCVB6CVTCTVNC2O5EHCXELNGBSSX3MXZW4WSNNTY66LUGWXFDQQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.095530806 (58h8m29.818093836s) [noise=true , address=104.36.82.8:4970 , piece_id=AWAB7RBRI4FHTPH6JJFF5PBIASTC35BEEBO6LEBHVN2VABJIKJ2A, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.096306831 (58h8m29.818869871s) [noise=true , address=45.91.116.9:4263 , piece_id=ERUFSJ4T6S4OKKD2HV3JJPHPYIY766EM6BZIRH2WLLZE7MTTNDYA, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.096325467 (58h8m29.818888507s) [noise=true , address=45.91.116.10:4081 , piece_id=GXZLER6ABRKZXJAOIDOBFX6GQ3DWUT6WEJ6GBVBYMARH5AYETZPQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.097367651 (58h8m29.819930701s) [noise=true , address=45.91.116.14:4190 , piece_id=GJDNIISBX57TZ5M2O6PNHCAU2Y42EBOGGZGSA53BEYEXGX2MSK6Q, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.098516289 (58h8m29.821079329s) [noise=true , address=104.36.82.13:4185 , piece_id=PLBDCXXBF2SAJA4MVSMP3EMXHBUJLBCKGNHKJKQADRGGML2PR7CQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.099428794 (58h8m29.821991844s) [noise=true , address=45.91.116.13:4155 , piece_id=RW23TJK745ZQQGOCS6WBAGNTLJHLMS6S75TQKQ32XSHIDU3K7JMQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.100729453 (58h8m29.823292493s) [noise=true , address=45.91.116.6:4974 , piece_id=NO5PTS2XBAJLDG63QM3WLTW42ILJCGCDHPNZTS7MFPSLG4S63SIA, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.102703189 (58h8m29.825266229s) [noise=true , address=45.91.116.5:4299 , piece_id=PKN2TOYC4MP32L4CLZTEDUWKCC4PAVPKRXMLRO3CMIKKM4R7NKRQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:21.830500633 (58h8m31.553063673s) [upload=55800]: Next returned nil segment Jul 26 21:44:21.830519369 (58h8m31.55308241s) [upload=55800]: Waiting for error group managing segments... Jul 26 21:44:22.116494557 (58h8m31.839057597s) [noise=true , address=45.91.116.6:4974 , piece_id=NO5PTS2XBAJLDG63QM3WLTW42ILJCGCDHPNZTS7MFPSLG4S63SIA, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.117448301 (58h8m31.840011341s) [noise=true , address=45.91.116.10:4082 , piece_id=DVKDGWK6ZH6TPG34U24OWOQAJSCT5CXIWZST6M7V5CUOEN4WQHPA, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.124989069 (58h8m31.847552099s) [noise=true , address=45.91.116.5:4299 , piece_id=PKN2TOYC4MP32L4CLZTEDUWKCC4PAVPKRXMLRO3CMIKKM4R7NKRQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.132441858 (58h8m31.855004928s) [noise=true , address=45.91.116.9:4262 , piece_id=4ZRP7DXDAAB4AXW6G4CBDONFRETI4I4JO3APCR4PU6OEUK3VBXLQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.139178869 (58h8m31.861741939s) [noise=true , address=45.91.116.14:4190 , piece_id=GJDNIISBX57TZ5M2O6PNHCAU2Y42EBOGGZGSA53BEYEXGX2MSK6Q, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.213497383 (58h8m31.936060423s) [noise=true , address=45.91.116.9:4263 , piece_id=ERUFSJ4T6S4OKKD2HV3JJPHPYIY766EM6BZIRH2WLLZE7MTTNDYA, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.228938817 (58h8m31.951501857s) [noise=true , address=45.91.116.13:4155 , piece_id=RW23TJK745ZQQGOCS6WBAGNTLJHLMS6S75TQKQ32XSHIDU3K7JMQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.236411715 (58h8m31.958974755s) [noise=true , address=45.91.116.10:4081 , piece_id=GXZLER6ABRKZXJAOIDOBFX6GQ3DWUT6WEJ6GBVBYMARH5AYETZPQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.311136446 (58h8m32.033699496s) [noise=true , address=45.91.116.5:4298 , piece_id=K3M3OTOPARCSNS2SQMULYQIQHTMMXAERDTOJIMG56AQPCCU42GQA, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:46.935082702 (58h8m56.657645742s) [noise=true , address=77.90.31.15:4697 , piece_id=42FVGSO2I7MS4BLUYQVP2XELP63BBADQ4RGZD6YJZ3RCUDQHIUTQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:47.003231763 (58h8m56.725794813s) [noise=true , address=77.90.31.16:4734 , piece_id=EVKGL35AJ4PC2VCOIHSJVW4HE7FWT2X3X2RI3UJNWMPCHPX5K2BA, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:49.222715854 (58h8m58.945278884s) [noise=true , address=77.90.31.17:4771 , piece_id=IBCVB6CVTCTVNC2O5EHCXELNGBSSX3MXZW4WSNNTY66LUGWXFDQQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:45438->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:45438->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->77.90.31.17:4771: use of closed network connection Jul 26 21:44:49.222775888 (58h8m58.945338928s) [noise=true , address=104.36.82.7:4912 , piece_id=XZ6GFCGJBG457FEG2COTVV3HG7YKJFK66ZSIHUET2BZ5YOWDWVOQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:104.36.82.7:4912): protocol: rpc: dial tcp 104.36.82.7:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:49.222833539 (58h8m58.945396579s) [noise=true , address=77.90.31.21:4877 , piece_id=AAWMVDGF5YY43Q5CJBS6TGT7JRAGBRG5HFIQIZFZP2RJVI7QJCMQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 188.166.225.170:37722->77.90.31.21:4877: use of closed network connection; write tcp 188.166.225.170:37722->77.90.31.21:4877: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37722->77.90.31.21:4877: use of closed network connection Jul 26 21:44:49.222876981 (58h8m58.945440021s) [noise=true , address=104.36.82.8:4970 , piece_id=AWAB7RBRI4FHTPH6JJFF5PBIASTC35BEEBO6LEBHVN2VABJIKJ2A, seg_pos={0 0}, upload=55800]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 188.166.225.170:47776->104.36.82.8:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:49.222904724 (58h8m58.945467764s) [noise=true , address=104.36.82.13:4185 , piece_id=PLBDCXXBF2SAJA4MVSMP3EMXHBUJLBCKGNHKJKQADRGGML2PR7CQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: rpc: dial tcp 104.36.82.13:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:49.222921206 (58h8m58.945484246s) [seg_pos={0 0}, upload=55800]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 21:44:49.451486612 (58h8m59.174049692s) [upload=55800]: Done uploading segments... Jul 26 21:44:49.451516990 (58h8m59.17408003s) [upload=55800]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 14.134807544s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 16 out of 60 is failed (threshold=3) [............................................XXXXXXXXXXXXXXXX] duration: 1m0.230239505s FAIL (2025-07-26 21:44:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 4) ======LOGS====== ... Jul 26 21:43:59.241310157 (58h8m8.963873187s) [noise=true , address=104.36.82.16:4325 , piece_id=7ETP77NYAWDF7UJ7LWGNUMKRTBOQESON7CQWWMNJKMADOQNZAUTQ, seg_pos={0 0}, upload=55796]: Uploading piece... Jul 26 21:43:59.241317471 (58h8m8.963880521s) [noise=true , address=45.91.116.10:4100 , piece_id=ZYBBFE45ZAQD2V4OAY7553D26Y2EMCV57B3JQDZNGLNYMO2OKTAQ, seg_pos={0 0}, upload=55796]: Uploading piece... Jul 26 21:43:59.241686797 (58h8m8.964249837s) [noise=true , address=104.36.82.54:4162 , piece_id=Y5QP7DT64TFEOYMTTIYLUKQZXANCMARLTKTOKICCGQGDFOEJ5CLQ, seg_pos={0 0}, upload=55796]: Uploading piece... Jul 26 21:43:59.242020586 (58h8m8.964583626s) [noise=true , address=104.36.82.8:4985 , piece_id=QSMRHF36WGHX3WSGC6WJIBE3JYIFWGQLE2KJBA5AHEDXGCAGCUNQ, seg_pos={0 0}, upload=55796]: Uploading piece... Jul 26 21:43:59.242184890 (58h8m8.96474793s) [noise=true , address=104.36.82.45:4925 , piece_id=VIO3XPCEJISNOQHLBVD3U5RKAZS7UQXI2NKY7OEVISRKVGJNX4IA, seg_pos={0 0}, upload=55796]: Uploading piece... Jul 26 21:43:59.242515813 (58h8m8.965078853s) [noise=true , address=79.127.201.215:4343 , piece_id=DZXIOT3CETKFH6ZQVNKAKT6G2OARKDLWIOERH53MVS22DSCALJDA, seg_pos={0 0}, upload=55796]: Uploading piece... Jul 26 21:43:59.242862697 (58h8m8.965425727s) [noise=true , address=104.36.82.39:4745 , piece_id=YVRTCLIZPL6QA3ZOZPNPKO57EUV3O2T33IGQJYHNOWQGRWGWSG2A, seg_pos={0 0}, upload=55796]: Uploading piece... Jul 26 21:43:59.242946337 (58h8m8.965509377s) [noise=true , address=64.49.13.140:4508 , piece_id=5Y65UMZC64QHNIEYCHFPBEZKNIFAAMI6Q3OVIB75PUBYPOQ273SQ, seg_pos={0 0}, upload=55796]: Uploading piece... Jul 26 21:43:59.243165347 (58h8m8.965728387s) [noise=true , address=104.36.82.55:4205 , piece_id=YX3EYQYA7G4NTGGE3VI62TBHGISSLSKLNV4QX2A75RHMZSK55DFA, seg_pos={0 0}, upload=55796]: Uploading piece... Jul 26 21:43:59.243358626 (58h8m8.965921656s) [noise=true , address=104.36.82.44:4904 , piece_id=XEJDGW5JG26PNRVQ2OG4FQRYVI2KCT5DFJLSZNYYKERXP2EEAGBQ, seg_pos={0 0}, upload=55796]: Uploading piece... Jul 26 21:43:59.243626359 (58h8m8.966189399s) [noise=true , address=104.36.82.52:4117 , piece_id=3T5LZ2SFVGJ5BIMCKJUOS4VJPXE4WZ7SXEY52Q2HXNB34AGVCVXA, seg_pos={0 0}, upload=55796]: Uploading piece... Jul 26 21:44:00.358193195 (58h8m10.080756225s) [upload=55796]: Next returned nil segment Jul 26 21:44:00.358210178 (58h8m10.080773218s) [upload=55796]: Waiting for error group managing segments... Jul 26 21:44:01.113548340 (58h8m10.83611137s) [noise=true , address=79.127.201.215:4343 , piece_id=DZXIOT3CETKFH6ZQVNKAKT6G2OARKDLWIOERH53MVS22DSCALJDA, seg_pos={0 0}, upload=55796]: Done uploading piece. err: <nil> Jul 26 21:44:01.274471786 (58h8m10.997034826s) [noise=true , address=79.127.205.245:4247 , piece_id=ZYA7EEMOCUMQNYBCU2HOMWJHJTS4LMXEGZG7YOMHO7XZXBQ7FNQQ, seg_pos={0 0}, upload=55796]: Done uploading piece. err: <nil> Jul 26 21:44:04.888365939 (58h8m14.610928969s) [noise=true , address=45.91.116.10:4100 , piece_id=ZYBBFE45ZAQD2V4OAY7553D26Y2EMCV57B3JQDZNGLNYMO2OKTAQ, seg_pos={0 0}, upload=55796]: Done uploading piece. err: <nil> Jul 26 21:44:07.548536256 (58h8m17.271099286s) [noise=true , address=64.49.13.140:4508 , piece_id=5Y65UMZC64QHNIEYCHFPBEZKNIFAAMI6Q3OVIB75PUBYPOQ273SQ, seg_pos={0 0}, upload=55796]: Done uploading piece. err: <nil> Jul 26 21:44:58.139135255 (58h9m7.861698285s) [noise=true , address=104.36.82.54:4162 , piece_id=Y5QP7DT64TFEOYMTTIYLUKQZXANCMARLTKTOKICCGQGDFOEJ5CLQ, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:12hV2HLvT4YUVqeM2eAVe8p8UFVsZK7MYN95YinC8KMss9sKDXt, address:104.36.82.54:4162): protocol: write tcp 188.166.225.170:41830->104.36.82.54:4162: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139180292 (58h9m7.861743332s) [noise=true , address=104.36.82.52:4117 , piece_id=3T5LZ2SFVGJ5BIMCKJUOS4VJPXE4WZ7SXEY52Q2HXNB34AGVCVXA, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:1QVEhFpjfLubGsZmoY1XQdDWevLy2xWdjtPq4JnbSYuBgEXgz7, address:104.36.82.52:4117): protocol: rpc: dial tcp 104.36.82.52:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139196804 (58h9m7.861759844s) [noise=true , address=104.36.82.39:4745 , piece_id=YVRTCLIZPL6QA3ZOZPNPKO57EUV3O2T33IGQJYHNOWQGRWGWSG2A, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:1ujPZUMvwigdNuqXMzvgaxXdiVQzEkfsSDNQY7BujAnHLHkXks, address:104.36.82.39:4745): protocol: rpc: dial tcp 104.36.82.39:4745: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139218516 (58h9m7.861781546s) [noise=true , address=104.36.82.9:4026 , piece_id=TE3P5JLSWG5MNPYTGOYNMKI7AJQHDMJKZOWIHS2BZR7OBJZL7PZQ, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:1M7HJBLoxHkRHXbPuDiqwxdb4EAq5wktfJY5FygdoDdgy59rRq, address:104.36.82.9:4026): protocol: rpc: dial tcp 104.36.82.9:4026: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139247391 (58h9m7.861810431s) [noise=true , address=104.36.82.40:4965 , piece_id=4EIZK5MYHCVB6VEOZONBKSKLGAB6CEVZYN5L4EAJZBGBZQTRVWDA, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:1ToFTqjRAWvpRQzf6vyHKDpforPyb4LbDGcCVhRhkwiBGKihnD, address:104.36.82.40:4965): protocol: rpc: dial tcp 104.36.82.40:4965: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139252551 (58h9m7.861815581s) [noise=true , address=104.36.82.46:4484 , piece_id=72HER4BJ2K3ARPZKQDWH535GWTYOBY53BAD4QN5CCQ7K42I2MBOA, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:1a5S5Ar7jNbK9MESghbWfmEgBPNSMN2tEnZeeG1kKzeX2dnSX3, address:104.36.82.46:4484): protocol: rpc: dial tcp 104.36.82.46:4484: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139284872 (58h9m7.861847912s) [noise=true , address=104.36.82.45:4925 , piece_id=VIO3XPCEJISNOQHLBVD3U5RKAZS7UQXI2NKY7OEVISRKVGJNX4IA, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:14ZdgenD4m2bMMmaHVyK3YivrzdSnrp72JvMJsk1uecW6KKFjn, address:104.36.82.45:4925): protocol: rpc: dial tcp 104.36.82.45:4925: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139294049 (58h9m7.86185708s) [noise=true , address=104.36.82.53:4139 , piece_id=KV6PVDPHIENG6WP7HZGMUVPX3BQUJRYHUPQW6BIMFWIIFDBOREMA, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:12T99BLYzWYXV5VkEKAYQVyzoeHA4XYz4Jrh8Dz3K6jqkec52Lg, address:104.36.82.53:4139): protocol: write tcp 188.166.225.170:33524->104.36.82.53:4139: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139316272 (58h9m7.861879312s) [noise=true , address=104.36.82.55:4205 , piece_id=YX3EYQYA7G4NTGGE3VI62TBHGISSLSKLNV4QX2A75RHMZSK55DFA, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:12UafxBVo9nuRVBcQmGE4gBP2pnXQ5YUrwHZdwPCeSBSBWCZKpQ, address:104.36.82.55:4205): protocol: rpc: dial tcp 104.36.82.55:4205: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139329849 (58h9m7.861892889s) [noise=true , address=104.36.82.16:4325 , piece_id=7ETP77NYAWDF7UJ7LWGNUMKRTBOQESON7CQWWMNJKMADOQNZAUTQ, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:1aTYNxPbXJGM29Xqovc5VK4cun64wr2DoduaVCvKDj38xCLGKJ, address:104.36.82.16:4325): protocol: rpc: dial tcp 104.36.82.16:4325: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139348844 (58h9m7.861911874s) [noise=true , address=104.36.82.8:4985 , piece_id=QSMRHF36WGHX3WSGC6WJIBE3JYIFWGQLE2KJBA5AHEDXGCAGCUNQ, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:12GvTsnY35oYVBUqHKk7eMWunjt8K6X2dZ9HhL4sGx24qV2ohFJ, address:104.36.82.8:4985): protocol: write tcp 188.166.225.170:52050->104.36.82.8:4985: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139360217 (58h9m7.861923257s) [noise=true , address=104.36.82.44:4904 , piece_id=XEJDGW5JG26PNRVQ2OG4FQRYVI2KCT5DFJLSZNYYKERXP2EEAGBQ, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:1dC7mC2Cbxmys7ukEkELyF9Us7F1qtptSP6yDJh1EfjVJ6uqsC, address:104.36.82.44:4904): protocol: rpc: dial tcp 104.36.82.44:4904: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139390735 (58h9m7.861953775s) [noise=true , address=104.36.82.10:4109 , piece_id=CV3ISH4S3YJ45ZK3JRH7PBU5VNPWE7HIA3YLQ3UJQRUQNFTABLHA, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:12ZE1ut9Tum4tzq2m8zbFaGUNLdNZSZQYARkayDAfE1b7uZtWjW, address:104.36.82.10:4109): protocol: rpc: dial tcp 104.36.82.10:4109: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139402748 (58h9m7.861965788s) [noise=true , address=104.36.82.47:4518 , piece_id=NVA2ESSFM7K7UCTCHSH7OLB334LIFBWFA6DTHI4Y7M6L7KOKNN7Q, seg_pos={0 0}, upload=55796]: Done uploading piece. err: ecclient: upload failed (node:17TZ9JLdCivFF46guhSTW6hcoUmTz5uo5wvLkDqC4BGmpm9eLb, address:104.36.82.47:4518): protocol: write tcp 188.166.225.170:52326->104.36.82.47:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:58.139421754 (58h9m7.861984794s) [seg_pos={0 0}, upload=55796]: Done waiting for segment. successful: 4 optimal: 14 errs: <nil> Jul 26 21:44:58.368020593 (58h9m8.090583634s) [upload=55796]: Done uploading segments... Jul 26 21:44:58.368044259 (58h9m8.090607299s) [upload=55796]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 4) Duration: 1m0.23025279s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X...................................XX.........XX.] duration: 16.964908009s FAIL (2025-07-26 21:44:57 UTC) orbiter warning: uplink: context deadline exceeded Duration: 16.964928609s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........................................X....X......XX....XX] duration: 30.230599848s FAIL (2025-07-26 21:44:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 15897 (58h8m17.709978927s) [upload=55790]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:44:07.987421177 (58h8m17.709984217s) [upload=55790]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:44:08.282987134 (58h8m18.005550174s) [upload=55790]: Done uploading segments... Jul 26 21:44:08.283006010 (58h8m18.005569051s) [upload=55790]: Upload finished. err: <nil> Jul 26 21:44:19.861345498 (58h8m29.583908538s) [upload=55800]: Starting upload Jul 26 21:44:19.861415974 (58h8m29.583979024s) [upload=55800]: Done starting upload Jul 26 21:44:19.861529861 (58h8m29.584092891s) [upload=55800]: Uploading segments... Jul 26 21:44:19.861610205 (58h8m29.584173235s) [upload=55800]: Got next segment. Inline: false Jul 26 21:44:19.861637628 (58h8m29.584200668s) [upload=55800]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:44:19.861641836 (58h8m29.584204876s) [upload=55800]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:44:20.094447584 (58h8m29.817010624s) [seg_pos={0 0}, upload=55800]: Begin upload segment... Jul 26 21:44:20.094481459 (58h8m29.817044499s) [seg_pos={0 0}, upload=55800]: Done begin upload segment. Jul 26 21:44:20.094499433 (58h8m29.817062473s) [noise=true , address=45.91.116.9:4262 , piece_id=4ZRP7DXDAAB4AXW6G4CBDONFRETI4I4JO3APCR4PU6OEUK3VBXLQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.094532116 (58h8m29.817095146s) [noise=true , address=77.90.31.16:4734 , piece_id=EVKGL35AJ4PC2VCOIHSJVW4HE7FWT2X3X2RI3UJNWMPCHPX5K2BA, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.094778667 (58h8m29.817341707s) [noise=true , address=45.91.116.5:4298 , piece_id=K3M3OTOPARCSNS2SQMULYQIQHTMMXAERDTOJIMG56AQPCCU42GQA, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.094935467 (58h8m29.817498497s) [noise=true , address=104.36.82.7:4912 , piece_id=XZ6GFCGJBG457FEG2COTVV3HG7YKJFK66ZSIHUET2BZ5YOWDWVOQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.095034998 (58h8m29.817598038s) [noise=true , address=77.90.31.15:4697 , piece_id=42FVGSO2I7MS4BLUYQVP2XELP63BBADQ4RGZD6YJZ3RCUDQHIUTQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.095218238 (58h8m29.817781279s) [noise=true , address=45.91.116.10:4082 , piece_id=DVKDGWK6ZH6TPG34U24OWOQAJSCT5CXIWZST6M7V5CUOEN4WQHPA, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.095277131 (58h8m29.817840171s) [noise=true , address=77.90.31.21:4877 , piece_id=AAWMVDGF5YY43Q5CJBS6TGT7JRAGBRG5HFIQIZFZP2RJVI7QJCMQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.095473016 (58h8m29.818036056s) [noise=true , address=77.90.31.17:4771 , piece_id=IBCVB6CVTCTVNC2O5EHCXELNGBSSX3MXZW4WSNNTY66LUGWXFDQQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.095530806 (58h8m29.818093836s) [noise=true , address=104.36.82.8:4970 , piece_id=AWAB7RBRI4FHTPH6JJFF5PBIASTC35BEEBO6LEBHVN2VABJIKJ2A, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.096306831 (58h8m29.818869871s) [noise=true , address=45.91.116.9:4263 , piece_id=ERUFSJ4T6S4OKKD2HV3JJPHPYIY766EM6BZIRH2WLLZE7MTTNDYA, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.096325467 (58h8m29.818888507s) [noise=true , address=45.91.116.10:4081 , piece_id=GXZLER6ABRKZXJAOIDOBFX6GQ3DWUT6WEJ6GBVBYMARH5AYETZPQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.097367651 (58h8m29.819930701s) [noise=true , address=45.91.116.14:4190 , piece_id=GJDNIISBX57TZ5M2O6PNHCAU2Y42EBOGGZGSA53BEYEXGX2MSK6Q, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.098516289 (58h8m29.821079329s) [noise=true , address=104.36.82.13:4185 , piece_id=PLBDCXXBF2SAJA4MVSMP3EMXHBUJLBCKGNHKJKQADRGGML2PR7CQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.099428794 (58h8m29.821991844s) [noise=true , address=45.91.116.13:4155 , piece_id=RW23TJK745ZQQGOCS6WBAGNTLJHLMS6S75TQKQ32XSHIDU3K7JMQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.100729453 (58h8m29.823292493s) [noise=true , address=45.91.116.6:4974 , piece_id=NO5PTS2XBAJLDG63QM3WLTW42ILJCGCDHPNZTS7MFPSLG4S63SIA, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:20.102703189 (58h8m29.825266229s) [noise=true , address=45.91.116.5:4299 , piece_id=PKN2TOYC4MP32L4CLZTEDUWKCC4PAVPKRXMLRO3CMIKKM4R7NKRQ, seg_pos={0 0}, upload=55800]: Uploading piece... Jul 26 21:44:21.830500633 (58h8m31.553063673s) [upload=55800]: Next returned nil segment Jul 26 21:44:21.830519369 (58h8m31.55308241s) [upload=55800]: Waiting for error group managing segments... Jul 26 21:44:22.116494557 (58h8m31.839057597s) [noise=true , address=45.91.116.6:4974 , piece_id=NO5PTS2XBAJLDG63QM3WLTW42ILJCGCDHPNZTS7MFPSLG4S63SIA, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.117448301 (58h8m31.840011341s) [noise=true , address=45.91.116.10:4082 , piece_id=DVKDGWK6ZH6TPG34U24OWOQAJSCT5CXIWZST6M7V5CUOEN4WQHPA, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.124989069 (58h8m31.847552099s) [noise=true , address=45.91.116.5:4299 , piece_id=PKN2TOYC4MP32L4CLZTEDUWKCC4PAVPKRXMLRO3CMIKKM4R7NKRQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.132441858 (58h8m31.855004928s) [noise=true , address=45.91.116.9:4262 , piece_id=4ZRP7DXDAAB4AXW6G4CBDONFRETI4I4JO3APCR4PU6OEUK3VBXLQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.139178869 (58h8m31.861741939s) [noise=true , address=45.91.116.14:4190 , piece_id=GJDNIISBX57TZ5M2O6PNHCAU2Y42EBOGGZGSA53BEYEXGX2MSK6Q, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.213497383 (58h8m31.936060423s) [noise=true , address=45.91.116.9:4263 , piece_id=ERUFSJ4T6S4OKKD2HV3JJPHPYIY766EM6BZIRH2WLLZE7MTTNDYA, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.228938817 (58h8m31.951501857s) [noise=true , address=45.91.116.13:4155 , piece_id=RW23TJK745ZQQGOCS6WBAGNTLJHLMS6S75TQKQ32XSHIDU3K7JMQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.236411715 (58h8m31.958974755s) [noise=true , address=45.91.116.10:4081 , piece_id=GXZLER6ABRKZXJAOIDOBFX6GQ3DWUT6WEJ6GBVBYMARH5AYETZPQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:22.311136446 (58h8m32.033699496s) [noise=true , address=45.91.116.5:4298 , piece_id=K3M3OTOPARCSNS2SQMULYQIQHTMMXAERDTOJIMG56AQPCCU42GQA, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:46.935082702 (58h8m56.657645742s) [noise=true , address=77.90.31.15:4697 , piece_id=42FVGSO2I7MS4BLUYQVP2XELP63BBADQ4RGZD6YJZ3RCUDQHIUTQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:47.003231763 (58h8m56.725794813s) [noise=true , address=77.90.31.16:4734 , piece_id=EVKGL35AJ4PC2VCOIHSJVW4HE7FWT2X3X2RI3UJNWMPCHPX5K2BA, seg_pos={0 0}, upload=55800]: Done uploading piece. err: <nil> Jul 26 21:44:49.222715854 (58h8m58.945278884s) [noise=true , address=77.90.31.17:4771 , piece_id=IBCVB6CVTCTVNC2O5EHCXELNGBSSX3MXZW4WSNNTY66LUGWXFDQQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: write tcp 188.166.225.170:45438->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:45438->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45438->77.90.31.17:4771: use of closed network connection Jul 26 21:44:49.222775888 (58h8m58.945338928s) [noise=true , address=104.36.82.7:4912 , piece_id=XZ6GFCGJBG457FEG2COTVV3HG7YKJFK66ZSIHUET2BZ5YOWDWVOQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:104.36.82.7:4912): protocol: rpc: dial tcp 104.36.82.7:4912: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:49.222833539 (58h8m58.945396579s) [noise=true , address=77.90.31.21:4877 , piece_id=AAWMVDGF5YY43Q5CJBS6TGT7JRAGBRG5HFIQIZFZP2RJVI7QJCMQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 188.166.225.170:37722->77.90.31.21:4877: use of closed network connection; write tcp 188.166.225.170:37722->77.90.31.21:4877: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37722->77.90.31.21:4877: use of closed network connection Jul 26 21:44:49.222876981 (58h8m58.945440021s) [noise=true , address=104.36.82.8:4970 , piece_id=AWAB7RBRI4FHTPH6JJFF5PBIASTC35BEEBO6LEBHVN2VABJIKJ2A, seg_pos={0 0}, upload=55800]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:104.36.82.8:4970): protocol: write tcp 188.166.225.170:47776->104.36.82.8:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:49.222904724 (58h8m58.945467764s) [noise=true , address=104.36.82.13:4185 , piece_id=PLBDCXXBF2SAJA4MVSMP3EMXHBUJLBCKGNHKJKQADRGGML2PR7CQ, seg_pos={0 0}, upload=55800]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:104.36.82.13:4185): protocol: rpc: dial tcp 104.36.82.13:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:49.222921206 (58h8m58.945484246s) [seg_pos={0 0}, upload=55800]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 21:44:49.451486612 (58h8m59.174049692s) [upload=55800]: Done uploading segments... Jul 26 21:44:49.451516990 (58h8m59.17408003s) [upload=55800]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.230624905s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..................X.........X.................X.............] duration: 1.889645469s FAIL (2025-07-26 21:44:31 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.88967669s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........X...................................XX.........XX] duration: 30.001156747s FAIL (2025-07-26 21:44:28 UTC) orbiter warning: uplink: context deadline exceeded Duration: 30.001181564s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........................................X....X......XX....X] duration: 30.002193728s FAIL (2025-07-26 21:44:09 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 0->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:35580->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35580->77.90.31.11:4554: use of closed network connection Jul 26 21:42:55.777398044 (58h7m5.499961074s) [seg_pos={0 0}, upload=55771]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:42:55.777496052 (58h7m5.500059092s) [upload=55771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:42:55.777520679 (58h7m5.500083719s) [upload=55771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:42:56.085044966 (58h7m5.807607996s) [upload=55771]: Done uploading segments... Jul 26 21:42:56.085064383 (58h7m5.807627423s) [upload=55771]: Upload finished. err: <nil> Jul 26 21:43:39.907045495 (58h7m49.629608525s) [upload=55790]: Starting upload Jul 26 21:43:39.907098717 (58h7m49.629661757s) [upload=55790]: Done starting upload Jul 26 21:43:39.907239706 (58h7m49.629802746s) [upload=55790]: Uploading segments... Jul 26 21:43:39.907334167 (58h7m49.629897207s) [upload=55790]: Got next segment. Inline: false Jul 26 21:43:39.907366418 (58h7m49.629929448s) [upload=55790]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:43:39.907371228 (58h7m49.629934268s) [upload=55790]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:43:40.147787537 (58h7m49.870350577s) [seg_pos={0 0}, upload=55790]: Begin upload segment... Jul 26 21:43:40.147850037 (58h7m49.870413077s) [seg_pos={0 0}, upload=55790]: Done begin upload segment. Jul 26 21:43:40.147872730 (58h7m49.87043576s) [noise=true , address=45.91.116.7:4010 , piece_id=FNC7P26O24UOKX5R2CPZF7WLGC5K5WGLZJ3ACCFRUKQJEM5DAFVA, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.147935912 (58h7m49.870498972s) [noise=true , address=45.91.116.5:4298 , piece_id=ITGM4F7IFRH4KIKXIE5DDM3P6JMXKM63HITOEP5GOPT7ZXO6KBIQ, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.148285700 (58h7m49.87084874s) [noise=true , address=104.36.82.15:4259 , piece_id=I33H54EBQSN4PRHR2LQKDRBLD2NXC6JHIT7LG7WHMFKEJ77ZWRSA, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.148580154 (58h7m49.871143194s) [noise=true , address=45.91.116.8:4048 , piece_id=7ISF25MJTGFPFNMY6LZYE4Y2O3WE3GRQXCLDX4CGDELAOCFIJU2A, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.148811095 (58h7m49.871374125s) [noise=true , address=45.91.116.11:4117 , piece_id=DMSAB3N7ICYEDNCWG5DRIDI7WFZKJXPED4ASKFSX6F6N6VU2QRJQ, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.148903042 (58h7m49.871466082s) [noise=true , address=77.90.31.17:4770 , piece_id=4FGIS6OX4NDCQQZ23UITSQVI3F524LSR7CROTY37PUA5UKYBN4FQ, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.149099938 (58h7m49.871662978s) [noise=true , address=45.91.116.4:4939 , piece_id=KZRF67AUGCTMPNYZKNHT5IQVO4XLDNY4JVA5L7QGYJTQQF6HFDGA, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.149156967 (58h7m49.871719997s) [noise=true , address=45.91.116.14:4190 , piece_id=QCGBY5UKGPSHJNRRNPWVSHGR6P4WRODWVTQV6E2MG3U535DQECGQ, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.149333675 (58h7m49.871896715s) [noise=true , address=104.36.82.15:4258 , piece_id=6W6MR2OXZOJOLLGCSZ6KMK6T3HTEV2HUP6Y7P2AC2UWWHDWCFVXA, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.149382138 (58h7m49.871945178s) [noise=true , address=104.36.82.7:4909 , piece_id=WAREOAZZCOD2FTTSCHBLOF3HGJRQQZPM2W4JSYEGVWXFJQX2WIZQ, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.149545821 (58h7m49.872108851s) [noise=true , address=45.91.116.10:4083 , piece_id=XALTSQLNVAFFOZ2JFR3EI3ECRX2QUHABWPALQ4YPUDKR5UFIX5AQ, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.149604583 (58h7m49.872167624s) [noise=true , address=45.91.116.6:4974 , piece_id=Y4JADCWAIFF7BQQV6NAQE4WC7PFKCY7AHIVMFP3Y23I5BBJI256Q, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.149902554 (58h7m49.872465594s) [noise=true , address=45.91.116.13:4155 , piece_id=CQNS5KM5OXWIO6YIKAX7OEKVMHY3YIU4JMXZXYCKU467ZXBQQ6NA, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.150144236 (58h7m49.872707276s) [noise=true , address=77.90.31.12:4589 , piece_id=UNZWTNNDTB5TX7EWNOU42GQYASTYLKMM63COE4L5HETRXBGINOWA, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.150334350 (58h7m49.87289739s) [noise=true , address=45.91.116.13:4154 , piece_id=JBD4YCTJ2TYANDMRYS7ZX7UBAH6IUYOVRA6GFVU4M4Z6ONBP2THQ, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:40.150530225 (58h7m49.873093265s) [noise=true , address=77.90.31.15:4698 , piece_id=GY6VUQDSUBAXCKEAVJSVGQCM4FJLFZAVHJCX5LTACEXLQGQG6IHQ, seg_pos={0 0}, upload=55790]: Uploading piece... Jul 26 21:43:41.905837172 (58h7m51.628400212s) [upload=55790]: Next returned nil segment Jul 26 21:43:41.905857672 (58h7m51.628420712s) [upload=55790]: Waiting for error group managing segments... Jul 26 21:43:42.181793331 (58h7m51.904356371s) [noise=true , address=45.91.116.13:4155 , piece_id=CQNS5KM5OXWIO6YIKAX7OEKVMHY3YIU4JMXZXYCKU467ZXBQQ6NA, seg_pos={0 0}, upload=55790]: Done uploading piece. err: <nil> Jul 26 21:43:42.256782198 (58h7m51.979345228s) [noise=true , address=45.91.116.13:4154 , piece_id=JBD4YCTJ2TYANDMRYS7ZX7UBAH6IUYOVRA6GFVU4M4Z6ONBP2THQ, seg_pos={0 0}, upload=55790]: Done uploading piece. err: <nil> Jul 26 21:43:42.257352178 (58h7m51.979915218s) [noise=true , address=45.91.116.7:4010 , piece_id=FNC7P26O24UOKX5R2CPZF7WLGC5K5WGLZJ3ACCFRUKQJEM5DAFVA, seg_pos={0 0}, upload=55790]: Done uploading piece. err: <nil> Jul 26 21:43:42.257605252 (58h7m51.980168292s) [noise=true , address=45.91.116.10:4083 , piece_id=XALTSQLNVAFFOZ2JFR3EI3ECRX2QUHABWPALQ4YPUDKR5UFIX5AQ, seg_pos={0 0}, upload=55790]: Done uploading piece. err: <nil> Jul 26 21:43:42.259050698 (58h7m51.981613738s) [noise=true , address=45.91.116.4:4939 , piece_id=KZRF67AUGCTMPNYZKNHT5IQVO4XLDNY4JVA5L7QGYJTQQF6HFDGA, seg_pos={0 0}, upload=55790]: Done uploading piece. err: <nil> Jul 26 21:43:42.268883170 (58h7m51.9914462s) [noise=true , address=45.91.116.5:4298 , piece_id=ITGM4F7IFRH4KIKXIE5DDM3P6JMXKM63HITOEP5GOPT7ZXO6KBIQ, seg_pos={0 0}, upload=55790]: Done uploading piece. err: <nil> Jul 26 21:43:42.314123038 (58h7m52.036686068s) [noise=true , address=45.91.116.14:4190 , piece_id=QCGBY5UKGPSHJNRRNPWVSHGR6P4WRODWVTQV6E2MG3U535DQECGQ, seg_pos={0 0}, upload=55790]: Done uploading piece. err: <nil> Jul 26 21:43:42.320778852 (58h7m52.043341913s) [noise=true , address=45.91.116.6:4974 , piece_id=Y4JADCWAIFF7BQQV6NAQE4WC7PFKCY7AHIVMFP3Y23I5BBJI256Q, seg_pos={0 0}, upload=55790]: Done uploading piece. err: <nil> Jul 26 21:43:42.325588274 (58h7m52.048151304s) [noise=true , address=45.91.116.8:4048 , piece_id=7ISF25MJTGFPFNMY6LZYE4Y2O3WE3GRQXCLDX4CGDELAOCFIJU2A, seg_pos={0 0}, upload=55790]: Done uploading piece. err: <nil> Jul 26 21:43:42.351061864 (58h7m52.073624904s) [noise=true , address=45.91.116.11:4117 , piece_id=DMSAB3N7ICYEDNCWG5DRIDI7WFZKJXPED4ASKFSX6F6N6VU2QRJQ, seg_pos={0 0}, upload=55790]: Done uploading piece. err: <nil> Jul 26 21:43:56.460981089 (58h8m6.183544129s) [noise=true , address=77.90.31.17:4770 , piece_id=4FGIS6OX4NDCQQZ23UITSQVI3F524LSR7CROTY37PUA5UKYBN4FQ, seg_pos={0 0}, upload=55790]: Done uploading piece. err: <nil> Jul 26 21:43:59.658425265 (58h8m9.380988305s) [noise=true , address=77.90.31.15:4698 , piece_id=GY6VUQDSUBAXCKEAVJSVGQCM4FJLFZAVHJCX5LTACEXLQGQG6IHQ, seg_pos={0 0}, upload=55790]: Done uploading piece. err: <nil> Jul 26 21:44:07.986913045 (58h8m17.709476085s) [noise=true , address=77.90.31.12:4589 , piece_id=UNZWTNNDTB5TX7EWNOU42GQYASTYLKMM63COE4L5HETRXBGINOWA, seg_pos={0 0}, upload=55790]: Done uploading piece. err: <nil> Jul 26 21:44:07.986946209 (58h8m17.709509269s) [seg_pos={0 0}, upload=55790]: Segment reached optimal threshold of 13 pieces. Jul 26 21:44:07.987296539 (58h8m17.709859579s) [noise=true , address=104.36.82.7:4909 , piece_id=WAREOAZZCOD2FTTSCHBLOF3HGJRQQZPM2W4JSYEGVWXFJQX2WIZQ, seg_pos={0 0}, upload=55790]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: rpc: dial tcp 104.36.82.7:4909: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:07.987302861 (58h8m17.709865901s) [noise=true , address=104.36.82.15:4258 , piece_id=6W6MR2OXZOJOLLGCSZ6KMK6T3HTEV2HUP6Y7P2AC2UWWHDWCFVXA, seg_pos={0 0}, upload=55790]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF): protocol: write tcp 188.166.225.170:44988->104.36.82.15:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:07.987337437 (58h8m17.709900477s) [noise=true , address=104.36.82.15:4259 , piece_id=I33H54EBQSN4PRHR2LQKDRBLD2NXC6JHIT7LG7WHMFKEJ77ZWRSA, seg_pos={0 0}, upload=55790]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:58988->104.36.82.15:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:44:07.987358257 (58h8m17.709921297s) [seg_pos={0 0}, upload=55790]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 26 21:44:07.987415897 (58h8m17.709978927s) [upload=55790]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 26 21:44:07.987421177 (58h8m17.709984217s) [upload=55790]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 26 21:44:08.282987134 (58h8m18.005550174s) [upload=55790]: Done uploading segments... Jul 26 21:44:08.283006010 (58h8m18.005569051s) [upload=55790]: Upload finished. err: <nil> Duration: 30.002213967s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...................X.........X.................X............] duration: 1.929424468s FAIL (2025-07-26 21:43:58 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.929455487s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 15 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXXXXXX] duration: 1m0.239317007s FAIL (2025-07-26 21:43:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 7) ======LOGS====== iece_id=GG4JHFWNS6OTCXRV455C74EXCVCWYVBENWXDTHBESECEIZFPYCEA, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:58.752034351 (58h7m8.474597391s) [noise=true , address=77.90.31.11:4572 , piece_id=K65AQNU2QS5TE33NGMYZBE23CK5B2R76XFYXDCH6QS2JY5L2JGZQ, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:58.752336098 (58h7m8.474899128s) [noise=true , address=79.127.201.215:4352 , piece_id=BKXBLZPYAVTVVKR67QNAQFRXRBB3ZQW6HJKOHJ5FZAPYOAO7NPCA, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:58.752364783 (58h7m8.474927843s) [noise=true , address=156.146.43.225:4449 , piece_id=XVC7LIDP446ZQFTYDU4L2CGD5GQUTR4WTOWGNBDVUSTUM3WOKK4Q, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:58.752571890 (58h7m8.47513493s) [noise=true , address=104.36.82.6:4890 , piece_id=QQ5H2U5AUC65EIXSSLSMI4OVSOBDT4JZRQ2C55J5T5LZYC3Q2BQQ, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:58.752636834 (58h7m8.475199874s) [noise=true , address=104.36.82.42:4831 , piece_id=XCCBASKPNR6ALYTLPYKUDSFUQG4ADNCGOZVODO3MQDQQB2ZL77EQ, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:58.752922741 (58h7m8.475485771s) [noise=true , address=104.36.82.47:4531 , piece_id=HWAQND6MHHXVSEFRWSKL275OJNB7V7PLBNQDRGZ57APW2LZF2XCA, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:58.753150998 (58h7m8.475714028s) [noise=true , address=104.36.82.53:4147 , piece_id=ICJFYOXZ5MBCQPBFWXHENFBLHRAQA2CTNCTJS6DJLQHGNDSNAHUA, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:58.753210451 (58h7m8.475773501s) [noise=true , address=104.36.82.22:4525 , piece_id=SEUHLPDP2N2FLA2NIMZNFPPRPBJYJKL4AHRN2NS4BFV2M5X4W2BQ, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:58.753366330 (58h7m8.47592937s) [noise=true , address=104.36.82.6:4880 , piece_id=KJ5FSUHMAYAJGGCZQUAYHPPCDLEP6YYDHNIRKIPF3A7VAVOZC4HA, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:58.753588294 (58h7m8.476151324s) [noise=true , address=79.127.220.101:4048 , piece_id=DPUUFJ2NAK7KPWAJEMF7R7CP36W3F6C3SKIFV2I4O47X5IRQGXSA, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:58.753852520 (58h7m8.47641556s) [noise=true , address=104.36.82.37:4672 , piece_id=2KPKPSKIK6B6JKU34P3KHGZRUB4QZV7YNHBOMK2B723JIFM7FXXA, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:58.754095374 (58h7m8.476658414s) [noise=true , address=79.127.205.234:4607 , piece_id=PDU4XSGZBRRN45WDLJHMA4SSKAVEOI6OSJ4UOWIHCOCMDVOMOHFA, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:58.754430406 (58h7m8.476993456s) [noise=true , address=104.36.82.46:4502 , piece_id=T34FGWECTDOOWHPOGZQ4BM7JFJT4Q2QHVFCWKFE35VIXPPHNJ6WQ, seg_pos={0 0}, upload=55782]: Uploading piece... Jul 26 21:42:59.877470103 (58h7m9.600033143s) [upload=55782]: Next returned nil segment Jul 26 21:42:59.877487487 (58h7m9.600050527s) [upload=55782]: Waiting for error group managing segments... Jul 26 21:43:00.191633715 (58h7m9.914196745s) [noise=true , address=79.127.205.234:4607 , piece_id=PDU4XSGZBRRN45WDLJHMA4SSKAVEOI6OSJ4UOWIHCOCMDVOMOHFA, seg_pos={0 0}, upload=55782]: Done uploading piece. err: <nil> Jul 26 21:43:00.646886060 (58h7m10.3694491s) [noise=true , address=79.127.201.215:4352 , piece_id=BKXBLZPYAVTVVKR67QNAQFRXRBB3ZQW6HJKOHJ5FZAPYOAO7NPCA, seg_pos={0 0}, upload=55782]: Done uploading piece. err: <nil> Jul 26 21:43:00.775373206 (58h7m10.497936246s) [noise=true , address=104.36.82.42:4831 , piece_id=XCCBASKPNR6ALYTLPYKUDSFUQG4ADNCGOZVODO3MQDQQB2ZL77EQ, seg_pos={0 0}, upload=55782]: Done uploading piece. err: <nil> Jul 26 21:43:00.827870730 (58h7m10.55043376s) [noise=true , address=156.146.43.225:4449 , piece_id=XVC7LIDP446ZQFTYDU4L2CGD5GQUTR4WTOWGNBDVUSTUM3WOKK4Q, seg_pos={0 0}, upload=55782]: Done uploading piece. err: <nil> Jul 26 21:43:03.565421514 (58h7m13.287984554s) [noise=true , address=79.127.220.101:4048 , piece_id=DPUUFJ2NAK7KPWAJEMF7R7CP36W3F6C3SKIFV2I4O47X5IRQGXSA, seg_pos={0 0}, upload=55782]: Done uploading piece. err: <nil> Jul 26 21:43:10.513329907 (58h7m20.235892947s) [noise=true , address=77.90.31.11:4572 , piece_id=K65AQNU2QS5TE33NGMYZBE23CK5B2R76XFYXDCH6QS2JY5L2JGZQ, seg_pos={0 0}, upload=55782]: Done uploading piece. err: <nil> Jul 26 21:43:24.786906844 (58h7m34.509469884s) [noise=true , address=104.36.82.9:4012 , piece_id=GG4JHFWNS6OTCXRV455C74EXCVCWYVBENWXDTHBESECEIZFPYCEA, seg_pos={0 0}, upload=55782]: Done uploading piece. err: <nil> Jul 26 21:43:57.666852028 (58h8m7.389415068s) [noise=true , address=104.36.82.6:4890 , piece_id=QQ5H2U5AUC65EIXSSLSMI4OVSOBDT4JZRQ2C55J5T5LZYC3Q2BQQ, seg_pos={0 0}, upload=55782]: Done uploading piece. err: ecclient: upload failed (node:1gakF2Cctzc6EhFScfLfBHJdcDHqMBcey4mNoToEwbtk8Pwix5, address:104.36.82.6:4890): protocol: write tcp 188.166.225.170:41128->104.36.82.6:4890: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:43:57.666922141 (58h8m7.389485181s) [noise=true , address=104.36.82.6:4880 , piece_id=KJ5FSUHMAYAJGGCZQUAYHPPCDLEP6YYDHNIRKIPF3A7VAVOZC4HA, seg_pos={0 0}, upload=55782]: Done uploading piece. err: ecclient: upload failed (node:125NsQhq7kRmTjKS8nuc51PZddApFEi5C38FzC5pEX2QrWYKzSW, address:104.36.82.6:4880): protocol: write tcp 188.166.225.170:47956->104.36.82.6:4880: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:43:57.666961006 (58h8m7.389524046s) [noise=true , address=104.36.82.37:4672 , piece_id=2KPKPSKIK6B6JKU34P3KHGZRUB4QZV7YNHBOMK2B723JIFM7FXXA, seg_pos={0 0}, upload=55782]: Done uploading piece. err: ecclient: upload failed (node:1G9dUkPkLRrEkMGsTiaN9iB8YnZ9GF7moCUPhGiu1XDDi8XnSu, address:104.36.82.37:4672): protocol: write tcp 188.166.225.170:47228->104.36.82.37:4672: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:43:57.666993969 (58h8m7.389557009s) [noise=true , address=104.36.82.47:4531 , piece_id=HWAQND6MHHXVSEFRWSKL275OJNB7V7PLBNQDRGZ57APW2LZF2XCA, seg_pos={0 0}, upload=55782]: Done uploading piece. err: ecclient: upload failed (node:12LZprfyqfD2YUwgAarzyjbgHEb6jMhAMAday5C45xk3j9RUS4u, address:104.36.82.47:4531): protocol: write tcp 188.166.225.170:35586->104.36.82.47:4531: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:43:57.667027713 (58h8m7.389590743s) [noise=true , address=104.36.82.46:4502 , piece_id=T34FGWECTDOOWHPOGZQ4BM7JFJT4Q2QHVFCWKFE35VIXPPHNJ6WQ, seg_pos={0 0}, upload=55782]: Done uploading piece. err: ecclient: upload failed (node:12HP5xuRAGgTsL11UQh7VRgfuFzwNdEMXgc8zPVv4QknStyv6Ta, address:104.36.82.46:4502): protocol: rpc: dial tcp 104.36.82.46:4502: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:43:57.667154797 (58h8m7.389717837s) [noise=true , address=104.36.82.22:4525 , piece_id=SEUHLPDP2N2FLA2NIMZNFPPRPBJYJKL4AHRN2NS4BFV2M5X4W2BQ, seg_pos={0 0}, upload=55782]: Done uploading piece. err: ecclient: upload failed (node:12Yx3qg5NAW8iAw56mkwALjkkhyFF5feKgTZxcTG31RxBsubh5A, address:104.36.82.22:4525): protocol: write tcp 188.166.225.170:55566->104.36.82.22:4525: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:43:57.667233528 (58h8m7.389796568s) [noise=true , address=104.36.82.5:4855 , piece_id=A6LW6ODLFWES4UCJQYDKXNGGCKRGD7LW7JNEJC2EZ72KHMO2KHHQ, seg_pos={0 0}, upload=55782]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: rpc: dial tcp 104.36.82.5:4855: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:43:57.667265629 (58h8m7.389828669s) [noise=true , address=104.36.82.53:4147 , piece_id=ICJFYOXZ5MBCQPBFWXHENFBLHRAQA2CTNCTJS6DJLQHGNDSNAHUA, seg_pos={0 0}, upload=55782]: Done uploading piece. err: ecclient: upload failed (node:12pg6fGWqRjS8Gf3zQgw8jbuytaCyMpJ2CAz4oRVHYdLAhttgnH, address:104.36.82.53:4147): protocol: write tcp 188.166.225.170:48758->104.36.82.53:4147: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:43:57.667281769 (58h8m7.389844809s) [noise=true , address=104.36.82.52:4088 , piece_id=RO7XZSZVARDKKSJU5UDLAFXDCKUE6PKNKFINZOD3JCAFP4NYHUGA, seg_pos={0 0}, upload=55782]: Done uploading piece. err: ecclient: upload failed (node:1284h6cPNVXJYMkSekGQVw8b7p89tkhpmgYLxTTYUDu2J7Wadoj, address:104.36.82.52:4088): protocol: write tcp 188.166.225.170:32794->104.36.82.52:4088: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:43:57.667318200 (58h8m7.38988124s) [noise=true , address=104.36.82.46:4481 , piece_id=AWDQI77QHC7RXFPQZG365QEJEIUSZADJALOTLR4GZRVILHLD3VJA, seg_pos={0 0}, upload=55782]: Done uploading piece. err: ecclient: upload failed (node:12bRK8WWmAnWUwzCfHu8JtZgrruYSJCvhnsYTKcaLccJcv5RQjp, address:104.36.82.46:4481): protocol: rpc: dial tcp 104.36.82.46:4481: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:43:57.667356513 (58h8m7.389919543s) [noise=true , address=104.36.82.54:4161 , piece_id=QLRGJTUYV542FPTZSF42VKGE4Z3ZCSZGKYKFNZRA73RDROTGZYOA, seg_pos={0 0}, upload=55782]: Done uploading piece. err: ecclient: upload failed (node:12pyhMcYjL5b9posZurgKXBnCHaMhaEjL3q3YWyc3aWkcYwbvTD, address:104.36.82.54:4161): protocol: write tcp 188.166.225.170:39316->104.36.82.54:4161: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:43:57.667391629 (58h8m7.38995466s) [seg_pos={0 0}, upload=55782]: Done waiting for segment. successful: 7 optimal: 14 errs: <nil> Jul 26 21:43:57.903892679 (58h8m7.626455719s) [upload=55782]: Done uploading segments... Jul 26 21:43:57.903927505 (58h8m7.626490545s) [upload=55782]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 7) Duration: 1m0.239329363s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [............X...................................XX.........X] duration: 30.230845253s FAIL (2025-07-26 21:43:18 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.230873576s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................................X....X......XX....] duration: 25.314685461s FAIL (2025-07-26 21:42:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== .605925663 (58h3m0.328488703s) [noise=true , address=104.36.82.5:4840 , piece_id=YBVUTJZ2RN2KMW2I4D5BCD42F62TQYRYBTNSY34U22267MOCPZJA, seg_pos={0 0}, upload=55710]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:104.36.82.5:4840): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:38:50.605944900 (58h3m0.32850794s) [seg_pos={0 0}, upload=55710]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:38:50.833093218 (58h3m0.555656258s) [upload=55710]: Done uploading segments... Jul 26 21:38:50.833138233 (58h3m0.555701293s) [upload=55710]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 26 21:38:56.916704322 (58h3m6.639267363s) [upload=55721]: Starting upload Jul 26 21:38:56.916756792 (58h3m6.639319832s) [upload=55721]: Done starting upload Jul 26 21:38:56.916934593 (58h3m6.639497633s) [upload=55721]: Uploading segments... Jul 26 21:38:56.916968998 (58h3m6.639532028s) [upload=55721]: Got next segment. Inline: false Jul 26 21:38:56.916992904 (58h3m6.639555934s) [upload=55721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:38:56.916997673 (58h3m6.639560713s) [upload=55721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:38:57.151780998 (58h3m6.874344038s) [seg_pos={0 0}, upload=55721]: Begin upload segment... Jul 26 21:38:57.151833869 (58h3m6.87439693s) [seg_pos={0 0}, upload=55721]: Done begin upload segment. Jul 26 21:38:57.151846042 (58h3m6.874409082s) [noise=true , address=104.36.82.10:4078 , piece_id=7XWQ27VFRDD74GLIA4NXVWB2AMUHLJE32V67OZLRMYO4S2633XVA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.151855912 (58h3m6.874418982s) [noise=true , address=77.90.31.13:4626 , piece_id=TOI5HBCROMG7GCHUDBEXSRPMNVDUP2CK6X2EN7TSSKPU6WGQ5N3Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.152377529 (58h3m6.874940569s) [noise=true , address=45.91.116.12:4225 , piece_id=64CCWISJYPKC56UKIJXR4YZF3QALQNFKQTFFY3LOAEI7SHWYMZAQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.152837629 (58h3m6.87540067s) [noise=true , address=104.36.82.9:4007 , piece_id=KVAKN67DEGIWE4F4H2DCJ24FLXYYWHAF5RKVLYJUYPHUNO5UUQNQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.153782999 (58h3m6.876346029s) [noise=true , address=45.91.116.9:4262 , piece_id=RORQB5KLKQST25MJH5LFIN6WMLFLUTWNNB7KYCABL4H2RKW5LA7Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154085317 (58h3m6.876648347s) [noise=true , address=104.36.82.7:4910 , piece_id=PVI2WNXDTO26APLOD5GM3Z6UR46T5WCJSNACIQHQRXXEUKSA5J6Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154213682 (58h3m6.876776722s) [noise=true , address=77.90.31.13:4628 , piece_id=YZ7W5C55YJ3W5DVX4R3M5DTO7KWZSHHPXOWPQ5GIKOPZGGK5ASOA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154448712 (58h3m6.877011752s) [noise=true , address=45.91.116.12:4227 , piece_id=2SX24CIJ6OW3NERBLVAQRRFVZ5KGLKBBODG26HUB7ITQFBPPMDPA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154666529 (58h3m6.877229569s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2VWFQAYUGU227JMDA3GPDQMFH3NJ4QLUV3RNOJ7OPKRABIEGVQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154892101 (58h3m6.877455141s) [noise=true , address=45.91.116.8:4047 , piece_id=VYOWMDIDWZOF4JVY47FYWF37K2C6GNS7W7ICV7FIXIKPA2K6R44A, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155080802 (58h3m6.877643842s) [noise=true , address=45.91.116.10:4081 , piece_id=65SFCLKX3UY7AWHTG4RZBDMWJVCPHF4OZPY5XNUWSZRJEPZZJJMQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155274713 (58h3m6.877837753s) [noise=true , address=45.91.116.11:4119 , piece_id=W6PB2GVX6MNC6X2COA4D75YWIMCB3X5EJJ4JZC47AC7V3T4J55ZA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155489684 (58h3m6.878052724s) [noise=true , address=104.36.82.7:4909 , piece_id=UY5FTAQRTGOBKH7LMYSH6WYH5XY3J5GXB65CKYUCAHNLHAW5ZUPA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155906972 (58h3m6.878470012s) [noise=true , address=45.91.116.7:4009 , piece_id=GKLQ7G7EVWCRCD6MQ4FDIJC2S27T6QKBSTJKEKHQ7UUNJOSL27JQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.156106574 (58h3m6.878669614s) [noise=true , address=104.36.82.3:4767 , piece_id=WSEOSBHEND7W3QF74Z44HF3S2QNI6EIF3GCJXCI7MMUF5DZ34RIA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.156330883 (58h3m6.878893923s) [noise=true , address=45.91.116.4:4937 , piece_id=Z2TBYYCFCGM7YGEN6QYABAZIDZLD2ENSVJ2Z7ETQBFQNLONB66XA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:58.895021080 (58h3m8.61758415s) [upload=55721]: Next returned nil segment Jul 26 21:38:58.895042832 (58h3m8.617605902s) [upload=55721]: Waiting for error group managing segments... Jul 26 21:38:59.168496886 (58h3m8.891059926s) [noise=true , address=45.91.116.12:4227 , piece_id=2SX24CIJ6OW3NERBLVAQRRFVZ5KGLKBBODG26HUB7ITQFBPPMDPA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.177587149 (58h3m8.900150189s) [noise=true , address=45.91.116.12:4225 , piece_id=64CCWISJYPKC56UKIJXR4YZF3QALQNFKQTFFY3LOAEI7SHWYMZAQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.256356711 (58h3m8.978919751s) [noise=true , address=45.91.116.11:4119 , piece_id=W6PB2GVX6MNC6X2COA4D75YWIMCB3X5EJJ4JZC47AC7V3T4J55ZA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.263143385 (58h3m8.985706425s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2VWFQAYUGU227JMDA3GPDQMFH3NJ4QLUV3RNOJ7OPKRABIEGVQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.277648448 (58h3m9.000211488s) [noise=true , address=45.91.116.4:4937 , piece_id=Z2TBYYCFCGM7YGEN6QYABAZIDZLD2ENSVJ2Z7ETQBFQNLONB66XA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.279279730 (58h3m9.00184277s) [noise=true , address=45.91.116.7:4009 , piece_id=GKLQ7G7EVWCRCD6MQ4FDIJC2S27T6QKBSTJKEKHQ7UUNJOSL27JQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.307593714 (58h3m9.030156754s) [noise=true , address=45.91.116.10:4081 , piece_id=65SFCLKX3UY7AWHTG4RZBDMWJVCPHF4OZPY5XNUWSZRJEPZZJJMQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.314667739 (58h3m9.037230779s) [noise=true , address=45.91.116.8:4047 , piece_id=VYOWMDIDWZOF4JVY47FYWF37K2C6GNS7W7ICV7FIXIKPA2K6R44A, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.349724434 (58h3m9.072287464s) [noise=true , address=45.91.116.9:4262 , piece_id=RORQB5KLKQST25MJH5LFIN6WMLFLUTWNNB7KYCABL4H2RKW5LA7Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:04.104309440 (58h3m13.82687248s) [noise=true , address=77.90.31.13:4626 , piece_id=TOI5HBCROMG7GCHUDBEXSRPMNVDUP2CK6X2EN7TSSKPU6WGQ5N3Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:05.457684029 (58h3m15.18024707s) [noise=true , address=77.90.31.13:4628 , piece_id=YZ7W5C55YJ3W5DVX4R3M5DTO7KWZSHHPXOWPQ5GIKOPZGGK5ASOA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:26.274464249 (58h3m35.997027289s) [noise=true , address=104.36.82.9:4007 , piece_id=KVAKN67DEGIWE4F4H2DCJ24FLXYYWHAF5RKVLYJUYPHUNO5UUQNQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: rpc: dial tcp 104.36.82.9:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274493555 (58h3m35.997056595s) [noise=true , address=104.36.82.7:4909 , piece_id=UY5FTAQRTGOBKH7LMYSH6WYH5XY3J5GXB65CKYUCAHNLHAW5ZUPA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 188.166.225.170:60166->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274509737 (58h3m35.997072777s) [noise=true , address=104.36.82.7:4910 , piece_id=PVI2WNXDTO26APLOD5GM3Z6UR46T5WCJSNACIQHQRXXEUKSA5J6Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274528824 (58h3m35.997091864s) [noise=true , address=104.36.82.3:4767 , piece_id=WSEOSBHEND7W3QF74Z44HF3S2QNI6EIF3GCJXCI7MMUF5DZ34RIA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: rpc: dial tcp 104.36.82.3:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274548020 (58h3m35.99711106s) [noise=true , address=104.36.82.10:4078 , piece_id=7XWQ27VFRDD74GLIA4NXVWB2AMUHLJE32V67OZLRMYO4S2633XVA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 188.166.225.170:56794->104.36.82.10:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274580142 (58h3m35.997143172s) [seg_pos={0 0}, upload=55721]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 21:39:26.511752419 (58h3m36.234315449s) [upload=55721]: Done uploading segments... Jul 26 21:39:26.511771585 (58h3m36.234334625s) [upload=55721]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 25.314710489s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration: 1m0.230893896s FAIL (2025-07-26 21:42:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 6) ======LOGS====== 26 21:41:58.295597503 (58h6m8.018160543s) [noise=true , address=79.127.201.215:4354 , piece_id=5P6OCLPTDCKQ4PJK63WIJMWZWKCUSH2JP7QJK2UW4VUHWQKNY6OQ, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:58.295768089 (58h6m8.018331129s) [noise=true , address=77.90.31.18:4831 , piece_id=DKCXYLWW2CPE5YGB7EZTIW6RHDPKOEQCJGABK566ZNPXDGDVSX5Q, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:58.296035441 (58h6m8.018598481s) [noise=true , address=104.36.82.42:4826 , piece_id=V7EPAWFOBUWOAKKGKHP2BNTWE5JXVR62BCYAKKT2XULEMUYFFGOQ, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:58.296340644 (58h6m8.018903684s) [noise=true , address=104.36.82.52:4118 , piece_id=ZB7QLQEYKCXKLFAGIVASXSDCBWE7AUPRSM7XLZ2O7RTC3NJFQPYA, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:58.296631090 (58h6m8.01919413s) [noise=true , address=79.127.205.235:4682 , piece_id=X4SNFWILFE6YW2YNYRWHASPOKGBIEGBWOJDS6WJ7FGKV465J4MOA, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:58.297553755 (58h6m8.020116795s) [noise=true , address=104.36.82.43:4861 , piece_id=EOR4AUVH7ICKTFBG7IP6YR4OE2UBRFGCEKBVEPMKIXKJKXC2A4XQ, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:58.298000169 (58h6m8.020563199s) [noise=true , address=79.127.205.245:4240 , piece_id=DUOAYNPLGOQJJ2I63TV374VS7EDXXW77Z73ETY4XYNIBL3WYCC5A, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:58.298948194 (58h6m8.021511234s) [noise=true , address=104.36.82.53:4144 , piece_id=T6JKQIOHV7WEBV64443PVEH6NEA76EKKLWFQGBOFYI2BU6SKRG3Q, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:58.299271132 (58h6m8.021834172s) [noise=true , address=104.36.82.35:4611 , piece_id=XYHPEMJ6FAH3QWOKIA5EUZWFW5XOIKAE3ZUAGQYTYS2547M5MNHA, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:58.299503406 (58h6m8.022066446s) [noise=true , address=64.49.13.138:4483 , piece_id=UA2PMGAFITXK64A6KCMY4OBGMPGSOPUU2BDHH52PLV7QQUSZAFQQ, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:58.300133932 (58h6m8.022696972s) [noise=true , address=104.36.82.47:4541 , piece_id=PJ3DVCKLSDVS7ORTHQJDAZ2MOLBNF6Z7HNBM4RZDKKOZQNENTYPQ, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:58.300397305 (58h6m8.022960346s) [noise=true , address=104.36.82.12:4178 , piece_id=ZRUJDJYL5MOOFDRLEZ3CNMCJLUMI55XNXERIKZLVHI5D72U5NURQ, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:58.300629671 (58h6m8.023192711s) [noise=true , address=104.36.82.55:4215 , piece_id=2KGCTKGZF6INLXLAIPDJTC5AA3RPPOHE2BESSFZLVRGKCIGGKENA, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:58.300886973 (58h6m8.023450013s) [noise=true , address=104.36.82.44:4888 , piece_id=SLPNX2USOLREQOCYAOI7QOPZMZ2DRMADSPEXL4MAZ257DKEKY6EA, seg_pos={0 0}, upload=55766]: Uploading piece... Jul 26 21:41:59.397614006 (58h6m9.120177046s) [upload=55766]: Next returned nil segment Jul 26 21:41:59.397632952 (58h6m9.120195982s) [upload=55766]: Waiting for error group managing segments... Jul 26 21:41:59.701027279 (58h6m9.423590319s) [noise=true , address=79.127.205.235:4682 , piece_id=X4SNFWILFE6YW2YNYRWHASPOKGBIEGBWOJDS6WJ7FGKV465J4MOA, seg_pos={0 0}, upload=55766]: Done uploading piece. err: <nil> Jul 26 21:41:59.726426626 (58h6m9.448989666s) [noise=true , address=79.127.205.245:4240 , piece_id=DUOAYNPLGOQJJ2I63TV374VS7EDXXW77Z73ETY4XYNIBL3WYCC5A, seg_pos={0 0}, upload=55766]: Done uploading piece. err: <nil> Jul 26 21:42:00.011071018 (58h6m9.733634058s) [noise=true , address=64.49.13.138:4483 , piece_id=UA2PMGAFITXK64A6KCMY4OBGMPGSOPUU2BDHH52PLV7QQUSZAFQQ, seg_pos={0 0}, upload=55766]: Done uploading piece. err: <nil> Jul 26 21:42:06.236697219 (58h6m15.959260259s) [noise=true , address=79.127.201.215:4354 , piece_id=5P6OCLPTDCKQ4PJK63WIJMWZWKCUSH2JP7QJK2UW4VUHWQKNY6OQ, seg_pos={0 0}, upload=55766]: Done uploading piece. err: <nil> Jul 26 21:42:08.271374127 (58h6m17.993937167s) [noise=true , address=77.90.31.18:4831 , piece_id=DKCXYLWW2CPE5YGB7EZTIW6RHDPKOEQCJGABK566ZNPXDGDVSX5Q, seg_pos={0 0}, upload=55766]: Done uploading piece. err: <nil> Jul 26 21:42:55.802357509 (58h7m5.524920549s) [noise=true , address=104.36.82.42:4826 , piece_id=V7EPAWFOBUWOAKKGKHP2BNTWE5JXVR62BCYAKKT2XULEMUYFFGOQ, seg_pos={0 0}, upload=55766]: Done uploading piece. err: <nil> Jul 26 21:42:57.208639335 (58h7m6.931202375s) [noise=true , address=104.36.82.12:4178 , piece_id=ZRUJDJYL5MOOFDRLEZ3CNMCJLUMI55XNXERIKZLVHI5D72U5NURQ, seg_pos={0 0}, upload=55766]: Done uploading piece. err: ecclient: upload failed (node:12LjJ3SmaNTQi8U1rPSGmR461UYC2CfMtoZ22SSV97hDuhMwC4V, address:104.36.82.12:4178): protocol: rpc: dial tcp 104.36.82.12:4178: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:42:57.209161494 (58h7m6.931724524s) [noise=true , address=104.36.82.52:4118 , piece_id=ZB7QLQEYKCXKLFAGIVASXSDCBWE7AUPRSM7XLZ2O7RTC3NJFQPYA, seg_pos={0 0}, upload=55766]: Done uploading piece. err: ecclient: upload failed (node:128w37UHxJB9R49iwmHbATt3bXUb1FMTQJ3Wjka7So7eqKWg4iF, address:104.36.82.52:4118): protocol: rpc: dial tcp 104.36.82.52:4118: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:42:57.209229395 (58h7m6.931792435s) [noise=true , address=104.36.82.53:4144 , piece_id=T6JKQIOHV7WEBV64443PVEH6NEA76EKKLWFQGBOFYI2BU6SKRG3Q, seg_pos={0 0}, upload=55766]: Done uploading piece. err: ecclient: upload failed (node:1FfzcPNLmNdrvoZBKcmCLz8d9rYSAnjFTo4NBnxBzN6dVcTNwt, address:104.36.82.53:4144): protocol: rpc: dial tcp 104.36.82.53:4144: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:42:57.209299969 (58h7m6.931863009s) [noise=true , address=104.36.82.44:4888 , piece_id=SLPNX2USOLREQOCYAOI7QOPZMZ2DRMADSPEXL4MAZ257DKEKY6EA, seg_pos={0 0}, upload=55766]: Done uploading piece. err: ecclient: upload failed (node:129FkuTqtaF3DmzHFvcCuJRvaubQV6HbaV1CnFtoY95QHH6NcME, address:104.36.82.44:4888): protocol: rpc: dial tcp 104.36.82.44:4888: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:42:57.209359132 (58h7m6.931922172s) [noise=true , address=104.36.82.46:4502 , piece_id=UL4FOAUY4L62JCRMUM73GVIFMKWGZLWHENPUVCI3IMFVFGUO57IA, seg_pos={0 0}, upload=55766]: Done uploading piece. err: ecclient: upload failed (node:12HP5xuRAGgTsL11UQh7VRgfuFzwNdEMXgc8zPVv4QknStyv6Ta, address:104.36.82.46:4502): protocol: rpc: dial tcp 104.36.82.46:4502: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:42:57.209393649 (58h7m6.931956699s) [noise=true , address=104.36.82.56:4234 , piece_id=MTRLCXRQFH42FYA6CMBTM6P7M37HUKMJ4QSXIFBDE3L5TZJAC4UQ, seg_pos={0 0}, upload=55766]: Done uploading piece. err: ecclient: upload failed (node:12uthP31WaBmn37h54Q6z9ZMtYRY6K8917Ms4kwyyADqu4EXPJy, address:104.36.82.56:4234): protocol: rpc: dial tcp 104.36.82.56:4234: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:42:57.209459144 (58h7m6.932022194s) [noise=true , address=104.36.82.16:4296 , piece_id=WYO6S3EYERYYR2TVKWBY26E6O42QFCT6FDGUB3Z6JAOYHG4YPGNA, seg_pos={0 0}, upload=55766]: Done uploading piece. err: ecclient: upload failed (node:12cKhDbA4ptwux2LYNPNAJ7Qh8jECsGWd1KWaK6T6pD38bJWdt1, address:104.36.82.16:4296): protocol: rpc: dial tcp 104.36.82.16:4296: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:42:57.209518658 (58h7m6.932081708s) [noise=true , address=104.36.82.47:4541 , piece_id=PJ3DVCKLSDVS7ORTHQJDAZ2MOLBNF6Z7HNBM4RZDKKOZQNENTYPQ, seg_pos={0 0}, upload=55766]: Done uploading piece. err: ecclient: upload failed (node:1m1ED9th2QhuoxBKEo3Dync9GpkWjGYNTSBHd57ud2BJoH3HXC, address:104.36.82.47:4541): protocol: rpc: dial tcp 104.36.82.47:4541: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:42:57.209550549 (58h7m6.932113589s) [noise=true , address=104.36.82.43:4861 , piece_id=EOR4AUVH7ICKTFBG7IP6YR4OE2UBRFGCEKBVEPMKIXKJKXC2A4XQ, seg_pos={0 0}, upload=55766]: Done uploading piece. err: ecclient: upload failed (node:122HC6BetMaBDNGEaHVwmadWtV1nCHx3vGBK2SWVDEQS1iBCz4o, address:104.36.82.43:4861): protocol: rpc: dial tcp 104.36.82.43:4861: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:42:57.209609702 (58h7m6.932172752s) [noise=true , address=104.36.82.35:4611 , piece_id=XYHPEMJ6FAH3QWOKIA5EUZWFW5XOIKAE3ZUAGQYTYS2547M5MNHA, seg_pos={0 0}, upload=55766]: Done uploading piece. err: ecclient: upload failed (node:1zkqEMSKx7zSHPFNZ51b5TLzL2cFQL8VyxAG14Lg1pT8zAu3tc, address:104.36.82.35:4611): protocol: rpc: dial tcp 104.36.82.35:4611: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:42:57.209644419 (58h7m6.932207459s) [noise=true , address=104.36.82.20:4438 , piece_id=TFJDL7PDZ3UEWHHKIMCWXLOFG66SJSHOMSV534WSPUAVH6KM7ZCA, seg_pos={0 0}, upload=55766]: Done uploading piece. err: ecclient: upload failed (node:1GEBfCYivHXYjSXHLmJjrGbgXh4UATzZBDzvx7oLTzeTHum8PT, address:104.36.82.20:4438): protocol: rpc: dial tcp 104.36.82.20:4438: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:42:57.209659026 (58h7m6.932222056s) [noise=true , address=104.36.82.55:4215 , piece_id=2KGCTKGZF6INLXLAIPDJTC5AA3RPPOHE2BESSFZLVRGKCIGGKENA, seg_pos={0 0}, upload=55766]: Done uploading piece. err: ecclient: upload failed (node:12Gbrbe3SdK7ccTugeFtmHE7gGWmLrZrGRXvb47EbiJPjfZfbKR, address:104.36.82.55:4215): protocol: rpc: dial tcp 104.36.82.55:4215: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:42:57.209687942 (58h7m6.932250972s) [seg_pos={0 0}, upload=55766]: Done waiting for segment. successful: 6 optimal: 14 errs: <nil> Jul 26 21:42:57.437523832 (58h7m7.160086872s) [upload=55766]: Done uploading segments... Jul 26 21:42:57.437548067 (58h7m7.160111107s) [upload=55766]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 6) Duration: 1m0.230907332s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [....................X.........X.................X...........] duration: 1.931619049s FAIL (2025-07-26 21:42:48 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.931660809s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................................X....X......XX...] duration: 26.413694639s FAIL (2025-07-26 21:42:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== .605925663 (58h3m0.328488703s) [noise=true , address=104.36.82.5:4840 , piece_id=YBVUTJZ2RN2KMW2I4D5BCD42F62TQYRYBTNSY34U22267MOCPZJA, seg_pos={0 0}, upload=55710]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:104.36.82.5:4840): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:38:50.605944900 (58h3m0.32850794s) [seg_pos={0 0}, upload=55710]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:38:50.833093218 (58h3m0.555656258s) [upload=55710]: Done uploading segments... Jul 26 21:38:50.833138233 (58h3m0.555701293s) [upload=55710]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 26 21:38:56.916704322 (58h3m6.639267363s) [upload=55721]: Starting upload Jul 26 21:38:56.916756792 (58h3m6.639319832s) [upload=55721]: Done starting upload Jul 26 21:38:56.916934593 (58h3m6.639497633s) [upload=55721]: Uploading segments... Jul 26 21:38:56.916968998 (58h3m6.639532028s) [upload=55721]: Got next segment. Inline: false Jul 26 21:38:56.916992904 (58h3m6.639555934s) [upload=55721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:38:56.916997673 (58h3m6.639560713s) [upload=55721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:38:57.151780998 (58h3m6.874344038s) [seg_pos={0 0}, upload=55721]: Begin upload segment... Jul 26 21:38:57.151833869 (58h3m6.87439693s) [seg_pos={0 0}, upload=55721]: Done begin upload segment. Jul 26 21:38:57.151846042 (58h3m6.874409082s) [noise=true , address=104.36.82.10:4078 , piece_id=7XWQ27VFRDD74GLIA4NXVWB2AMUHLJE32V67OZLRMYO4S2633XVA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.151855912 (58h3m6.874418982s) [noise=true , address=77.90.31.13:4626 , piece_id=TOI5HBCROMG7GCHUDBEXSRPMNVDUP2CK6X2EN7TSSKPU6WGQ5N3Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.152377529 (58h3m6.874940569s) [noise=true , address=45.91.116.12:4225 , piece_id=64CCWISJYPKC56UKIJXR4YZF3QALQNFKQTFFY3LOAEI7SHWYMZAQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.152837629 (58h3m6.87540067s) [noise=true , address=104.36.82.9:4007 , piece_id=KVAKN67DEGIWE4F4H2DCJ24FLXYYWHAF5RKVLYJUYPHUNO5UUQNQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.153782999 (58h3m6.876346029s) [noise=true , address=45.91.116.9:4262 , piece_id=RORQB5KLKQST25MJH5LFIN6WMLFLUTWNNB7KYCABL4H2RKW5LA7Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154085317 (58h3m6.876648347s) [noise=true , address=104.36.82.7:4910 , piece_id=PVI2WNXDTO26APLOD5GM3Z6UR46T5WCJSNACIQHQRXXEUKSA5J6Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154213682 (58h3m6.876776722s) [noise=true , address=77.90.31.13:4628 , piece_id=YZ7W5C55YJ3W5DVX4R3M5DTO7KWZSHHPXOWPQ5GIKOPZGGK5ASOA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154448712 (58h3m6.877011752s) [noise=true , address=45.91.116.12:4227 , piece_id=2SX24CIJ6OW3NERBLVAQRRFVZ5KGLKBBODG26HUB7ITQFBPPMDPA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154666529 (58h3m6.877229569s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2VWFQAYUGU227JMDA3GPDQMFH3NJ4QLUV3RNOJ7OPKRABIEGVQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154892101 (58h3m6.877455141s) [noise=true , address=45.91.116.8:4047 , piece_id=VYOWMDIDWZOF4JVY47FYWF37K2C6GNS7W7ICV7FIXIKPA2K6R44A, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155080802 (58h3m6.877643842s) [noise=true , address=45.91.116.10:4081 , piece_id=65SFCLKX3UY7AWHTG4RZBDMWJVCPHF4OZPY5XNUWSZRJEPZZJJMQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155274713 (58h3m6.877837753s) [noise=true , address=45.91.116.11:4119 , piece_id=W6PB2GVX6MNC6X2COA4D75YWIMCB3X5EJJ4JZC47AC7V3T4J55ZA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155489684 (58h3m6.878052724s) [noise=true , address=104.36.82.7:4909 , piece_id=UY5FTAQRTGOBKH7LMYSH6WYH5XY3J5GXB65CKYUCAHNLHAW5ZUPA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155906972 (58h3m6.878470012s) [noise=true , address=45.91.116.7:4009 , piece_id=GKLQ7G7EVWCRCD6MQ4FDIJC2S27T6QKBSTJKEKHQ7UUNJOSL27JQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.156106574 (58h3m6.878669614s) [noise=true , address=104.36.82.3:4767 , piece_id=WSEOSBHEND7W3QF74Z44HF3S2QNI6EIF3GCJXCI7MMUF5DZ34RIA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.156330883 (58h3m6.878893923s) [noise=true , address=45.91.116.4:4937 , piece_id=Z2TBYYCFCGM7YGEN6QYABAZIDZLD2ENSVJ2Z7ETQBFQNLONB66XA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:58.895021080 (58h3m8.61758415s) [upload=55721]: Next returned nil segment Jul 26 21:38:58.895042832 (58h3m8.617605902s) [upload=55721]: Waiting for error group managing segments... Jul 26 21:38:59.168496886 (58h3m8.891059926s) [noise=true , address=45.91.116.12:4227 , piece_id=2SX24CIJ6OW3NERBLVAQRRFVZ5KGLKBBODG26HUB7ITQFBPPMDPA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.177587149 (58h3m8.900150189s) [noise=true , address=45.91.116.12:4225 , piece_id=64CCWISJYPKC56UKIJXR4YZF3QALQNFKQTFFY3LOAEI7SHWYMZAQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.256356711 (58h3m8.978919751s) [noise=true , address=45.91.116.11:4119 , piece_id=W6PB2GVX6MNC6X2COA4D75YWIMCB3X5EJJ4JZC47AC7V3T4J55ZA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.263143385 (58h3m8.985706425s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2VWFQAYUGU227JMDA3GPDQMFH3NJ4QLUV3RNOJ7OPKRABIEGVQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.277648448 (58h3m9.000211488s) [noise=true , address=45.91.116.4:4937 , piece_id=Z2TBYYCFCGM7YGEN6QYABAZIDZLD2ENSVJ2Z7ETQBFQNLONB66XA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.279279730 (58h3m9.00184277s) [noise=true , address=45.91.116.7:4009 , piece_id=GKLQ7G7EVWCRCD6MQ4FDIJC2S27T6QKBSTJKEKHQ7UUNJOSL27JQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.307593714 (58h3m9.030156754s) [noise=true , address=45.91.116.10:4081 , piece_id=65SFCLKX3UY7AWHTG4RZBDMWJVCPHF4OZPY5XNUWSZRJEPZZJJMQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.314667739 (58h3m9.037230779s) [noise=true , address=45.91.116.8:4047 , piece_id=VYOWMDIDWZOF4JVY47FYWF37K2C6GNS7W7ICV7FIXIKPA2K6R44A, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.349724434 (58h3m9.072287464s) [noise=true , address=45.91.116.9:4262 , piece_id=RORQB5KLKQST25MJH5LFIN6WMLFLUTWNNB7KYCABL4H2RKW5LA7Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:04.104309440 (58h3m13.82687248s) [noise=true , address=77.90.31.13:4626 , piece_id=TOI5HBCROMG7GCHUDBEXSRPMNVDUP2CK6X2EN7TSSKPU6WGQ5N3Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:05.457684029 (58h3m15.18024707s) [noise=true , address=77.90.31.13:4628 , piece_id=YZ7W5C55YJ3W5DVX4R3M5DTO7KWZSHHPXOWPQ5GIKOPZGGK5ASOA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:26.274464249 (58h3m35.997027289s) [noise=true , address=104.36.82.9:4007 , piece_id=KVAKN67DEGIWE4F4H2DCJ24FLXYYWHAF5RKVLYJUYPHUNO5UUQNQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: rpc: dial tcp 104.36.82.9:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274493555 (58h3m35.997056595s) [noise=true , address=104.36.82.7:4909 , piece_id=UY5FTAQRTGOBKH7LMYSH6WYH5XY3J5GXB65CKYUCAHNLHAW5ZUPA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 188.166.225.170:60166->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274509737 (58h3m35.997072777s) [noise=true , address=104.36.82.7:4910 , piece_id=PVI2WNXDTO26APLOD5GM3Z6UR46T5WCJSNACIQHQRXXEUKSA5J6Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274528824 (58h3m35.997091864s) [noise=true , address=104.36.82.3:4767 , piece_id=WSEOSBHEND7W3QF74Z44HF3S2QNI6EIF3GCJXCI7MMUF5DZ34RIA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: rpc: dial tcp 104.36.82.3:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274548020 (58h3m35.99711106s) [noise=true , address=104.36.82.10:4078 , piece_id=7XWQ27VFRDD74GLIA4NXVWB2AMUHLJE32V67OZLRMYO4S2633XVA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 188.166.225.170:56794->104.36.82.10:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274580142 (58h3m35.997143172s) [seg_pos={0 0}, upload=55721]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 21:39:26.511752419 (58h3m36.234315449s) [upload=55721]: Done uploading segments... Jul 26 21:39:26.511771585 (58h3m36.234334625s) [upload=55721]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 26.413724024s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X...................................XX.........] duration: 13.897991279s FAIL (2025-07-26 21:42:08 UTC) orbiter warning: uplink: failed to upload enough pieces (needed at least 13 but got 12) Duration: 13.898017279s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration: 1m0.233007897s FAIL (2025-07-26 21:41:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== ad=55750]: Uploading piece... Jul 26 21:40:57.851117085 (58h5m7.573680125s) [noise=true , address=79.127.205.235:4676 , piece_id=3GHMZ6CGNFEJ4XH3G5V7FIQN2C7SGPRD5L2EVKEVJQSAOH6VLQRA, seg_pos={0 0}, upload=55750]: Uploading piece... Jul 26 21:40:57.851183491 (58h5m7.573746531s) [noise=true , address=104.36.82.52:4115 , piece_id=RNUHUB4XTVEKFTA2E7WLN5HZNTVQ2SAMAKEUCQDD56VJYSAPGJYA, seg_pos={0 0}, upload=55750]: Uploading piece... Jul 26 21:40:57.851439782 (58h5m7.574002823s) [noise=true , address=79.127.201.215:4357 , piece_id=4G2NLI3MB327U2Z2WUYLX4COS35L2JZRHX72UNEPKA3IPZSUQ66Q, seg_pos={0 0}, upload=55750]: Uploading piece... Jul 26 21:40:57.851517852 (58h5m7.574080892s) [noise=true , address=104.36.82.7:4935 , piece_id=DY2ZHWGS5YBG6WNZZQII7F26HJ7FBXCNP4VU75R2PNC6LVO72EYQ, seg_pos={0 0}, upload=55750]: Uploading piece... Jul 26 21:40:57.851826673 (58h5m7.574389713s) [noise=true , address=77.90.31.17:4793 , piece_id=I4TGX46XVRZI2JQXNLX3WFV3OWLKO3KXURARWRNDJ276BHKUUC6Q, seg_pos={0 0}, upload=55750]: Uploading piece... Jul 26 21:40:57.851911946 (58h5m7.574474986s) [noise=true , address=104.36.82.3:4775 , piece_id=3DMDCSJARNAZTT6AWPYGVSSWBJKL2HODGU4MYBNNXNLWGCI7RUHQ, seg_pos={0 0}, upload=55750]: Uploading piece... Jul 26 21:40:57.852088934 (58h5m7.574651964s) [noise=true , address=104.36.82.56:4231 , piece_id=CV7L4XEQRG2ODFHGWUYD5XOPJYQBYRG4EM4PRRYSV3CBMYQU3R2A, seg_pos={0 0}, upload=55750]: Uploading piece... Jul 26 21:40:57.852117458 (58h5m7.574680498s) [noise=true , address=104.36.82.54:4168 , piece_id=LSOXNBKPLBYRNJXQZ2YIMN5W2UNVC75Z453LL7QCZ7YDAWVQ7MBQ, seg_pos={0 0}, upload=55750]: Uploading piece... Jul 26 21:40:57.852352509 (58h5m7.574915549s) [noise=true , address=104.36.82.38:4696 , piece_id=JGBQMTTYH42RET6VSQHGEAPB3KTTHBSW6OT4ODHDFB7SJAEL6S4Q, seg_pos={0 0}, upload=55750]: Uploading piece... Jul 26 21:40:57.852554184 (58h5m7.575117224s) [noise=true , address=104.36.82.47:4529 , piece_id=5HODIBFLLH3HHSYYROH6ITSJSO25FY5FMVU44WK5SLOKFZJTJKHA, seg_pos={0 0}, upload=55750]: Uploading piece... Jul 26 21:40:57.852749488 (58h5m7.575312528s) [noise=true , address=104.36.82.2:4751 , piece_id=NJVRADYUNQRAN57NWCGS7POF3WNC62BKHIUCNO3HXMS5PN3EC3TQ, seg_pos={0 0}, upload=55750]: Uploading piece... Jul 26 21:40:57.852937417 (58h5m7.575500447s) [noise=true , address=104.36.82.34:4577 , piece_id=YH534L4GJ5SNJ7NZUJPE7YIVJ773ANZDH4YVV53G6WGF5EYWZPYQ, seg_pos={0 0}, upload=55750]: Uploading piece... Jul 26 21:40:58.962827975 (58h5m8.685391015s) [upload=55750]: Next returned nil segment Jul 26 21:40:58.962853524 (58h5m8.685416564s) [upload=55750]: Waiting for error group managing segments... Jul 26 21:40:59.261536094 (58h5m8.984099134s) [noise=true , address=79.127.205.235:4676 , piece_id=3GHMZ6CGNFEJ4XH3G5V7FIQN2C7SGPRD5L2EVKEVJQSAOH6VLQRA, seg_pos={0 0}, upload=55750]: Done uploading piece. err: <nil> Jul 26 21:40:59.769845476 (58h5m9.492408516s) [noise=true , address=79.127.201.215:4357 , piece_id=4G2NLI3MB327U2Z2WUYLX4COS35L2JZRHX72UNEPKA3IPZSUQ66Q, seg_pos={0 0}, upload=55750]: Done uploading piece. err: <nil> Jul 26 21:41:03.210072814 (58h5m12.932635894s) [noise=true , address=45.91.116.9:4269 , piece_id=DUBXSX55EXQOL233BB3VBCAN5E3H7CCOHGS6T7H7N3UT7M2RXQNQ, seg_pos={0 0}, upload=55750]: Done uploading piece. err: <nil> Jul 26 21:41:04.081115663 (58h5m13.803678703s) [noise=true , address=77.90.31.17:4793 , piece_id=I4TGX46XVRZI2JQXNLX3WFV3OWLKO3KXURARWRNDJ276BHKUUC6Q, seg_pos={0 0}, upload=55750]: Done uploading piece. err: <nil> Jul 26 21:41:14.718618067 (58h5m24.441181098s) [noise=true , address=104.36.82.38:4696 , piece_id=JGBQMTTYH42RET6VSQHGEAPB3KTTHBSW6OT4ODHDFB7SJAEL6S4Q, seg_pos={0 0}, upload=55750]: Done uploading piece. err: <nil> Jul 26 21:41:56.746631277 (58h6m6.469194317s) [noise=true , address=104.36.82.47:4529 , piece_id=5HODIBFLLH3HHSYYROH6ITSJSO25FY5FMVU44WK5SLOKFZJTJKHA, seg_pos={0 0}, upload=55750]: Done uploading piece. err: ecclient: upload failed (node:1m7FrQEps7d8enRYk4v17UxcDs8o8L2CFcVEyB3zR4EniXrYeH, address:104.36.82.47:4529): protocol: rpc: dial tcp 104.36.82.47:4529: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:41:56.746719877 (58h6m6.469282917s) [noise=true , address=104.36.82.19:4410 , piece_id=CKZHMVZVVFYBC2ZYFLBBTRK4HC3ANGSGW4VEH4OW6NFFT5TUFPNA, seg_pos={0 0}, upload=55750]: Done uploading piece. err: ecclient: upload failed (node:12hLhHpBmtWRfwU7osDv82ncUNjTXWR4JQfpPxtoLKMNyeaUSSA, address:104.36.82.19:4410): protocol: rpc: dial tcp 104.36.82.19:4410: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:41:56.746750545 (58h6m6.469313585s) [noise=true , address=104.36.82.7:4935 , piece_id=DY2ZHWGS5YBG6WNZZQII7F26HJ7FBXCNP4VU75R2PNC6LVO72EYQ, seg_pos={0 0}, upload=55750]: Done uploading piece. err: ecclient: upload failed (node:17YmVUymv5Cx9wFw1g67CfEHxg8B75TcFkTCRpNdTm2PHanHVJ, address:104.36.82.7:4935): protocol: rpc: dial tcp 104.36.82.7:4935: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:41:56.746791494 (58h6m6.469354534s) [noise=true , address=104.36.82.56:4231 , piece_id=CV7L4XEQRG2ODFHGWUYD5XOPJYQBYRG4EM4PRRYSV3CBMYQU3R2A, seg_pos={0 0}, upload=55750]: Done uploading piece. err: ecclient: upload failed (node:129WWyyn7SrJFbFJ2kjPVhwDNRoYM7Kj449TZiZLzUbqpwK7qJ4, address:104.36.82.56:4231): protocol: write tcp 188.166.225.170:44314->104.36.82.56:4231: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:41:56.746826561 (58h6m6.469389601s) [noise=true , address=104.36.82.3:4775 , piece_id=3DMDCSJARNAZTT6AWPYGVSSWBJKL2HODGU4MYBNNXNLWGCI7RUHQ, seg_pos={0 0}, upload=55750]: Done uploading piece. err: ecclient: upload failed (node:12BvQLLnX5A2BgKeqn1ubs5iBeCUsUGH8hFjpB2HvY53YH9Nw1n, address:104.36.82.3:4775): protocol: rpc: dial tcp 104.36.82.3:4775: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:41:56.746850507 (58h6m6.469413547s) [noise=true , address=104.36.82.34:4577 , piece_id=YH534L4GJ5SNJ7NZUJPE7YIVJ773ANZDH4YVV53G6WGF5EYWZPYQ, seg_pos={0 0}, upload=55750]: Done uploading piece. err: ecclient: upload failed (node:129EkvQCWxahFtpWNwEviNgeT9HutToW6yt2T9wEkRi29gcFaUP, address:104.36.82.34:4577): protocol: write tcp 188.166.225.170:43814->104.36.82.34:4577: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:41:56.746903428 (58h6m6.469466468s) [noise=true , address=104.36.82.40:4968 , piece_id=EPZ2QIBUZ7X4TL2PVIASD534HNNVNOS2JELEUDCKQERCHRXJCA6Q, seg_pos={0 0}, upload=55750]: Done uploading piece. err: ecclient: upload failed (node:1J98jLqn1SVMYjTYxKnNqYbRxSQ5vaUnq2G7XpVngHTaWRJ8oX, address:104.36.82.40:4968): protocol: rpc: dial tcp 104.36.82.40:4968: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:41:56.746919679 (58h6m6.469482719s) [noise=true , address=104.36.82.54:4168 , piece_id=LSOXNBKPLBYRNJXQZ2YIMN5W2UNVC75Z453LL7QCZ7YDAWVQ7MBQ, seg_pos={0 0}, upload=55750]: Done uploading piece. err: ecclient: upload failed (node:1XH68Yk8yh2Qa8vcPRYAuLHdGR7Jy4KtSBKYKgwAJpdDvVu7up, address:104.36.82.54:4168): protocol: write tcp 188.166.225.170:38274->104.36.82.54:4168: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:41:56.746960678 (58h6m6.469523718s) [noise=true , address=104.36.82.49:4005 , piece_id=FXFZN3S4QPESJ3S2UGJXU7ZZ757T6GGMUYWNBJVVX3H7UF5KDN3A, seg_pos={0 0}, upload=55750]: Done uploading piece. err: ecclient: upload failed (node:1AivThSVLAp9zneNK79sic4AqnTQ62yvNTFF36imigarMGqDSW, address:104.36.82.49:4005): protocol: rpc: dial tcp 104.36.82.49:4005: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:41:56.746994452 (58h6m6.469557492s) [noise=true , address=104.36.82.52:4115 , piece_id=RNUHUB4XTVEKFTA2E7WLN5HZNTVQ2SAMAKEUCQDD56VJYSAPGJYA, seg_pos={0 0}, upload=55750]: Done uploading piece. err: ecclient: upload failed (node:1ikQ2b343QztY3vBFQ3cmEwEEFszpB2TPMtstufHdFfMEkJkBV, address:104.36.82.52:4115): protocol: write tcp 188.166.225.170:57222->104.36.82.52:4115: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:41:56.747002928 (58h6m6.469565958s) [noise=true , address=104.36.82.43:4848 , piece_id=6KK34SUQ46SRSE5VGEITDO3RP7CRI5W7HN3O2I5VMZKMUL3ZN4AA, seg_pos={0 0}, upload=55750]: Done uploading piece. err: ecclient: upload failed (node:1DPLv5Kfsx9RLVWeRrxbdkaLGJc4dfrzGnPLwc8us9hT5uiPbj, address:104.36.82.43:4848): protocol: rpc: dial tcp 104.36.82.43:4848: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:41:56.747019590 (58h6m6.46958263s) [noise=true , address=104.36.82.2:4751 , piece_id=NJVRADYUNQRAN57NWCGS7POF3WNC62BKHIUCNO3HXMS5PN3EC3TQ, seg_pos={0 0}, upload=55750]: Done uploading piece. err: ecclient: upload failed (node:1rJkr2qSmcxGv7smeBBLwVjLoWz3Wsd7CvWWc3B7mzP5dBG3Uz, address:104.36.82.2:4751): protocol: rpc: dial tcp 104.36.82.2:4751: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:41:56.747029900 (58h6m6.46959294s) [noise=true , address=104.36.82.46:4484 , piece_id=NRGO3RXFSFPS2QZCGKFSGLGFLO2EWKK6WBTNQF5ZBNLLVWDIQYXA, seg_pos={0 0}, upload=55750]: Done uploading piece. err: ecclient: upload failed (node:1a5S5Ar7jNbK9MESghbWfmEgBPNSMN2tEnZeeG1kKzeX2dnSX3, address:104.36.82.46:4484): protocol: rpc: dial tcp 104.36.82.46:4484: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:41:56.747044298 (58h6m6.469607368s) [seg_pos={0 0}, upload=55750]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Jul 26 21:41:56.977861090 (58h6m6.70042412s) [upload=55750]: Done uploading segments... Jul 26 21:41:56.977897991 (58h6m6.700461031s) [upload=55750]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 1m0.23302461s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....................X.........X.................X..........] duration: 1.879602586s FAIL (2025-07-26 21:41:38 UTC) orbiter warning: digest mismatch: after 0783c6a051cf3b535b7c61f8e6f0cfe751970c7121d9bce2182fd51e29ca8fc6 != before ee59801cb45aa958945ed73fcee20d9c3604e7f558c75f49df119aca36a29d88 Duration: 1.879619869s core uplink us1 vivint
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................................X....X......XX..] duration: 13.71862317s FAIL (2025-07-26 21:41:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== .605925663 (58h3m0.328488703s) [noise=true , address=104.36.82.5:4840 , piece_id=YBVUTJZ2RN2KMW2I4D5BCD42F62TQYRYBTNSY34U22267MOCPZJA, seg_pos={0 0}, upload=55710]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:104.36.82.5:4840): protocol: context deadline exceeded; context deadline exceeded Jul 26 21:38:50.605944900 (58h3m0.32850794s) [seg_pos={0 0}, upload=55710]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 26 21:38:50.833093218 (58h3m0.555656258s) [upload=55710]: Done uploading segments... Jul 26 21:38:50.833138233 (58h3m0.555701293s) [upload=55710]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Jul 26 21:38:56.916704322 (58h3m6.639267363s) [upload=55721]: Starting upload Jul 26 21:38:56.916756792 (58h3m6.639319832s) [upload=55721]: Done starting upload Jul 26 21:38:56.916934593 (58h3m6.639497633s) [upload=55721]: Uploading segments... Jul 26 21:38:56.916968998 (58h3m6.639532028s) [upload=55721]: Got next segment. Inline: false Jul 26 21:38:56.916992904 (58h3m6.639555934s) [upload=55721]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 26 21:38:56.916997673 (58h3m6.639560713s) [upload=55721]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 26 21:38:57.151780998 (58h3m6.874344038s) [seg_pos={0 0}, upload=55721]: Begin upload segment... Jul 26 21:38:57.151833869 (58h3m6.87439693s) [seg_pos={0 0}, upload=55721]: Done begin upload segment. Jul 26 21:38:57.151846042 (58h3m6.874409082s) [noise=true , address=104.36.82.10:4078 , piece_id=7XWQ27VFRDD74GLIA4NXVWB2AMUHLJE32V67OZLRMYO4S2633XVA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.151855912 (58h3m6.874418982s) [noise=true , address=77.90.31.13:4626 , piece_id=TOI5HBCROMG7GCHUDBEXSRPMNVDUP2CK6X2EN7TSSKPU6WGQ5N3Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.152377529 (58h3m6.874940569s) [noise=true , address=45.91.116.12:4225 , piece_id=64CCWISJYPKC56UKIJXR4YZF3QALQNFKQTFFY3LOAEI7SHWYMZAQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.152837629 (58h3m6.87540067s) [noise=true , address=104.36.82.9:4007 , piece_id=KVAKN67DEGIWE4F4H2DCJ24FLXYYWHAF5RKVLYJUYPHUNO5UUQNQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.153782999 (58h3m6.876346029s) [noise=true , address=45.91.116.9:4262 , piece_id=RORQB5KLKQST25MJH5LFIN6WMLFLUTWNNB7KYCABL4H2RKW5LA7Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154085317 (58h3m6.876648347s) [noise=true , address=104.36.82.7:4910 , piece_id=PVI2WNXDTO26APLOD5GM3Z6UR46T5WCJSNACIQHQRXXEUKSA5J6Q, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154213682 (58h3m6.876776722s) [noise=true , address=77.90.31.13:4628 , piece_id=YZ7W5C55YJ3W5DVX4R3M5DTO7KWZSHHPXOWPQ5GIKOPZGGK5ASOA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154448712 (58h3m6.877011752s) [noise=true , address=45.91.116.12:4227 , piece_id=2SX24CIJ6OW3NERBLVAQRRFVZ5KGLKBBODG26HUB7ITQFBPPMDPA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154666529 (58h3m6.877229569s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2VWFQAYUGU227JMDA3GPDQMFH3NJ4QLUV3RNOJ7OPKRABIEGVQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.154892101 (58h3m6.877455141s) [noise=true , address=45.91.116.8:4047 , piece_id=VYOWMDIDWZOF4JVY47FYWF37K2C6GNS7W7ICV7FIXIKPA2K6R44A, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155080802 (58h3m6.877643842s) [noise=true , address=45.91.116.10:4081 , piece_id=65SFCLKX3UY7AWHTG4RZBDMWJVCPHF4OZPY5XNUWSZRJEPZZJJMQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155274713 (58h3m6.877837753s) [noise=true , address=45.91.116.11:4119 , piece_id=W6PB2GVX6MNC6X2COA4D75YWIMCB3X5EJJ4JZC47AC7V3T4J55ZA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155489684 (58h3m6.878052724s) [noise=true , address=104.36.82.7:4909 , piece_id=UY5FTAQRTGOBKH7LMYSH6WYH5XY3J5GXB65CKYUCAHNLHAW5ZUPA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.155906972 (58h3m6.878470012s) [noise=true , address=45.91.116.7:4009 , piece_id=GKLQ7G7EVWCRCD6MQ4FDIJC2S27T6QKBSTJKEKHQ7UUNJOSL27JQ, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.156106574 (58h3m6.878669614s) [noise=true , address=104.36.82.3:4767 , piece_id=WSEOSBHEND7W3QF74Z44HF3S2QNI6EIF3GCJXCI7MMUF5DZ34RIA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:57.156330883 (58h3m6.878893923s) [noise=true , address=45.91.116.4:4937 , piece_id=Z2TBYYCFCGM7YGEN6QYABAZIDZLD2ENSVJ2Z7ETQBFQNLONB66XA, seg_pos={0 0}, upload=55721]: Uploading piece... Jul 26 21:38:58.895021080 (58h3m8.61758415s) [upload=55721]: Next returned nil segment Jul 26 21:38:58.895042832 (58h3m8.617605902s) [upload=55721]: Waiting for error group managing segments... Jul 26 21:38:59.168496886 (58h3m8.891059926s) [noise=true , address=45.91.116.12:4227 , piece_id=2SX24CIJ6OW3NERBLVAQRRFVZ5KGLKBBODG26HUB7ITQFBPPMDPA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.177587149 (58h3m8.900150189s) [noise=true , address=45.91.116.12:4225 , piece_id=64CCWISJYPKC56UKIJXR4YZF3QALQNFKQTFFY3LOAEI7SHWYMZAQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.256356711 (58h3m8.978919751s) [noise=true , address=45.91.116.11:4119 , piece_id=W6PB2GVX6MNC6X2COA4D75YWIMCB3X5EJJ4JZC47AC7V3T4J55ZA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.263143385 (58h3m8.985706425s) [noise=true , address=45.91.116.11:4117 , piece_id=BW2VWFQAYUGU227JMDA3GPDQMFH3NJ4QLUV3RNOJ7OPKRABIEGVQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.277648448 (58h3m9.000211488s) [noise=true , address=45.91.116.4:4937 , piece_id=Z2TBYYCFCGM7YGEN6QYABAZIDZLD2ENSVJ2Z7ETQBFQNLONB66XA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.279279730 (58h3m9.00184277s) [noise=true , address=45.91.116.7:4009 , piece_id=GKLQ7G7EVWCRCD6MQ4FDIJC2S27T6QKBSTJKEKHQ7UUNJOSL27JQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.307593714 (58h3m9.030156754s) [noise=true , address=45.91.116.10:4081 , piece_id=65SFCLKX3UY7AWHTG4RZBDMWJVCPHF4OZPY5XNUWSZRJEPZZJJMQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.314667739 (58h3m9.037230779s) [noise=true , address=45.91.116.8:4047 , piece_id=VYOWMDIDWZOF4JVY47FYWF37K2C6GNS7W7ICV7FIXIKPA2K6R44A, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:38:59.349724434 (58h3m9.072287464s) [noise=true , address=45.91.116.9:4262 , piece_id=RORQB5KLKQST25MJH5LFIN6WMLFLUTWNNB7KYCABL4H2RKW5LA7Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:04.104309440 (58h3m13.82687248s) [noise=true , address=77.90.31.13:4626 , piece_id=TOI5HBCROMG7GCHUDBEXSRPMNVDUP2CK6X2EN7TSSKPU6WGQ5N3Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:05.457684029 (58h3m15.18024707s) [noise=true , address=77.90.31.13:4628 , piece_id=YZ7W5C55YJ3W5DVX4R3M5DTO7KWZSHHPXOWPQ5GIKOPZGGK5ASOA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: <nil> Jul 26 21:39:26.274464249 (58h3m35.997027289s) [noise=true , address=104.36.82.9:4007 , piece_id=KVAKN67DEGIWE4F4H2DCJ24FLXYYWHAF5RKVLYJUYPHUNO5UUQNQ, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:104.36.82.9:4007): protocol: rpc: dial tcp 104.36.82.9:4007: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274493555 (58h3m35.997056595s) [noise=true , address=104.36.82.7:4909 , piece_id=UY5FTAQRTGOBKH7LMYSH6WYH5XY3J5GXB65CKYUCAHNLHAW5ZUPA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:104.36.82.7:4909): protocol: write tcp 188.166.225.170:60166->104.36.82.7:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274509737 (58h3m35.997072777s) [noise=true , address=104.36.82.7:4910 , piece_id=PVI2WNXDTO26APLOD5GM3Z6UR46T5WCJSNACIQHQRXXEUKSA5J6Q, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:104.36.82.7:4910): protocol: rpc: dial tcp 104.36.82.7:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274528824 (58h3m35.997091864s) [noise=true , address=104.36.82.3:4767 , piece_id=WSEOSBHEND7W3QF74Z44HF3S2QNI6EIF3GCJXCI7MMUF5DZ34RIA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:104.36.82.3:4767): protocol: rpc: dial tcp 104.36.82.3:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274548020 (58h3m35.99711106s) [noise=true , address=104.36.82.10:4078 , piece_id=7XWQ27VFRDD74GLIA4NXVWB2AMUHLJE32V67OZLRMYO4S2633XVA, seg_pos={0 0}, upload=55721]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:104.36.82.10:4078): protocol: write tcp 188.166.225.170:56794->104.36.82.10:4078: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 26 21:39:26.274580142 (58h3m35.997143172s) [seg_pos={0 0}, upload=55721]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 26 21:39:26.511752419 (58h3m36.234315449s) [upload=55721]: Done uploading segments... Jul 26 21:39:26.511771585 (58h3m36.234334625s) [upload=55721]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 13.718647837s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uplink", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)