Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.......................................X.X......XX.X........] duration: 5.89488864s FAIL (2025-07-18 02:28:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.894914739s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................XX.......XXXX.........] duration: 6.102044182s FAIL (2025-07-18 02:28:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.102066586s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [........................................X.X......XX.X.......] duration: 5.658671214s FAIL (2025-07-18 02:27:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.658704028s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......................................XX.......XXXX........] duration: 6.067785877s FAIL (2025-07-18 02:27:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.067811507s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.........................................X.X......XX.X......] duration: 5.740678647s FAIL (2025-07-18 02:26:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.740705759s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........................................XX.......XXXX.......] duration: 5.889449844s FAIL (2025-07-18 02:26:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.889481775s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........................................XX.......XXXX......] duration: 5.997985586s FAIL (2025-07-18 02:25:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.998007318s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [..........................................X.X......XX.X.....] duration: 5.988997752s FAIL (2025-07-18 02:25:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.989024884s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [...........................................X.X......XX.X....] duration: 5.914924435s FAIL (2025-07-18 02:24:27 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.914953752s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........................................XX.......XXXX.....] duration: 5.854599173s FAIL (2025-07-18 02:24:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.854632977s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [............................................X.X......XX.X...] duration: 5.92894359s FAIL (2025-07-18 02:23:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.928969388s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........................................XX.......XXXX....] duration: 6.466813929s FAIL (2025-07-18 02:23:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.466836462s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.............................................X.X......XX.X..] duration: 5.768475494s FAIL (2025-07-18 02:23:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.768496915s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............................................XX.......XXXX...] duration: 6.009675593s FAIL (2025-07-18 02:23:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.009695041s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [..............................................X.X......XX.X.] duration: 5.860523589s FAIL (2025-07-18 02:22:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.860545201s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [...............................................X.X......XX.X] duration: 1m0.252722322s FAIL (2025-07-18 02:22:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 8 02:21:35.177845305 (593h11m9.94848664s) [upload=474424]: Uploading segments... Jul 18 02:21:35.177987377 (593h11m9.948628712s) [upload=474424]: Got next segment. Inline: false Jul 18 02:21:35.178033806 (593h11m9.948675141s) [upload=474424]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:21:35.178038314 (593h11m9.948679649s) [upload=474424]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:21:35.659204645 (593h11m10.42984598s) [seg_pos={0 0}, upload=474424]: Begin upload segment... Jul 18 02:21:35.659248218 (593h11m10.429889543s) [seg_pos={0 0}, upload=474424]: Done begin upload segment. Jul 18 02:21:35.659273166 (593h11m10.429914501s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659300989 (593h11m10.429942324s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659629969 (593h11m10.430271294s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.659876430 (593h11m10.430517765s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660107403 (593h11m10.430748728s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660254844 (593h11m10.43089618s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660432375 (593h11m10.43107371s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660528989 (593h11m10.431170324s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660882566 (593h11m10.431523901s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.660963921 (593h11m10.431605256s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.661213188 (593h11m10.431854513s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.662765930 (593h11m10.433407265s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.663721818 (593h11m10.434363153s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665003170 (593h11m10.435644505s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.665753345 (593h11m10.43639468s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666249704 (593h11m10.436891039s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666615976 (593h11m10.437257311s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:35.666965143 (593h11m10.437606478s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Uploading piece... Jul 18 02:21:36.771152997 (593h11m11.541794332s) [upload=474424]: Next returned nil segment Jul 18 02:21:36.771200578 (593h11m11.541841923s) [upload=474424]: Waiting for error group managing segments... Jul 18 02:21:37.062727209 (593h11m11.833368544s) [noise=true , address=79.127.205.233:4625 , piece_id=WE43MEX62LE3ZHIAOSUJMZZGVOEV2YJN342OJIFRTKLXOI3B3O6A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.234454516 (593h11m12.005095851s) [noise=true , address=79.127.205.235:4673 , piece_id=6K354YXA3H7QJ6W5Y3MWVI353R5YWR6KEJNBBN7UTWIZ7U4AGZ5A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.356875997 (593h11m12.127517372s) [noise=true , address=45.91.95.4:4019 , piece_id=ZA4HDNHYRQSJ55SLOORN2JZCEWAO3TW6BLF4VUTEDTXWVKSGUBDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.415013602 (593h11m12.185654937s) [noise=true , address=45.91.95.2:4022 , piece_id=5DZT7U4BNJRNETPUPGUTMJBFOLDHTZEKFKEPGZI6MPBIQ42MWMDQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.481780768 (593h11m12.252422104s) [noise=true , address=45.91.95.3:4025 , piece_id=OOIXS37H6IYGW7OQFAWVERFDHATX2UASL2QDCIMS5ED2KQPAITKQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.587656953 (593h11m12.358298288s) [noise=true , address=45.91.116.13:4158 , piece_id=VKPPGYBEY2EK7TVTBOCALAZTNJLTX7SXZ23IMA5AZBZYULR7C4RA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.603489976 (593h11m12.374131311s) [noise=true , address=104.36.82.51:4077 , piece_id=P6667ZOVEU66GZA3VHBSZCVRVU3GGEZ6ICWPPGTHXBSZICDIXK2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.632675457 (593h11m12.403316793s) [noise=true , address=45.91.116.6:4001 , piece_id=FAHE4JPUNMAMDXQNDSO26DDXGVGVNQYTAMG7XEAWE3UB5C5JLF7Q, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.633584979 (593h11m12.404226314s) [noise=true , address=79.127.219.38:4021 , piece_id=PVRMXGJHHJQDDSDBTARCNOUOOORY4MZ4524K6LSHI6MNAEBMGFFA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.662224003 (593h11m12.432865338s) [noise=true , address=79.127.219.51:4214 , piece_id=74IMTXLXEVONC6ABLITL3YDEZMOV7TWUZKAWIQBXNXJ7LFJI6TOA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.676693477 (593h11m12.447334802s) [noise=true , address=104.36.82.5:4854 , piece_id=SETPOSULUVRAGK5AB7YUTY6GZJVBAFITMIOI33IPZT6SYJ6UUGFQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.704001817 (593h11m12.474643152s) [noise=true , address=104.36.82.49:4991 , piece_id=WJX5D3GLMRQBLTIYW4CLVKVV22N6W4HR4653KFOHSJ5VPGW33OWQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:21:37.717466809 (593h11m12.488108144s) [noise=true , address=104.36.82.11:4143 , piece_id=FVVMO7VL4UC33C2FL2UA63LCDRUMSOLUEVQTOPVOBXVXTRZ7QRPA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: <nil> Jul 18 02:22:34.473113950 (593h12m9.243755285s) [noise=true , address=77.90.31.12:4602 , piece_id=OS5CDYJA4T3TDZHCHI6U2IWGAAGXYOZS22NXWA526GXKQVLK6E2A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1GtnQbBbyjZQHrAXhG5r4Kt4MYyiUtEgRnj8VsSjA2q4bGkyjf, address:77.90.31.12:4602): protocol: rpc: dial tcp 77.90.31.12:4602: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473169977 (593h12m9.243811312s) [noise=true , address=77.90.31.10:4539 , piece_id=WIMAODSQ3QRV7Y7SZN76RAYCARUP32RRQGE2RQNW4KHOPEYSBT4A, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:1DYHaVii53Q4FhtYNBMC64MP1uKHffSTMdfETK8kS94e4ZPfka, address:77.90.31.10:4539): protocol: write tcp 188.166.225.170:59550->77.90.31.10:4539: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473232436 (593h12m9.243873761s) [noise=true , address=77.90.31.21:4887 , piece_id=SGP53S4FGVOF5KTS465L25MJZDYFDGDDAJANNPLU3BSIDPYDO4BA, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12hE2JH2wY7TepnbCVpSxkTcVifqui3QK42tLh1neU2BxZ2BRFJ, address:77.90.31.21:4887): protocol: rpc: dial tcp 77.90.31.21:4887: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473258586 (593h12m9.243899921s) [noise=true , address=77.90.31.18:4840 , piece_id=5WSE2YR4UYZZCP5O4HHF4AFDAEFZTMWNBCC36NH42HOUD2ASJNUQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12HcxQABJsUpjxiYdAkRFTpRTrT4KfrP8RKZkUhsyea1EKmwd5V, address:77.90.31.18:4840): protocol: rpc: dial tcp 77.90.31.18:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:22:34.473359119 (593h12m9.244000454s) [noise=true , address=77.90.31.17:4797 , piece_id=OHAWDT4NLGT7URIDDE3NOROCLQCAICQF6RTQNMNJBCFUGETK6DMQ, seg_pos={0 0}, upload=474424]: Done uploading piece. err: ecclient: upload failed (node:12FNxEAEnP3DqdDb16XaBCdmoqdUPEmMtLHDoTuQDBukr2nQWDW, address:77.90.31.17:4797): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:22:34.473402903 (593h12m9.244044238s) [seg_pos={0 0}, upload=474424]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Jul 18 02:22:34.723019907 (593h12m9.493661232s) [upload=474424]: Done uploading segments... Jul 18 02:22:34.723045165 (593h12m9.4936865s) [upload=474424]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.252764031s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............................................XX.......XXXX..] duration: 6.042211487s FAIL (2025-07-18 02:22:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.042234171s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............................................XX.......XXXX.] duration: 6.049026083s FAIL (2025-07-18 02:21:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.049058143s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [................................................X.X......XX.] duration: 5.877952986s FAIL (2025-07-18 02:21:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== egin Jul 18 02:20:00.064079005 (593h9m34.83472034s) [upload=474403]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:00.553474341 (593h9m35.324115676s) [seg_pos={0 0}, upload=474403]: Begin upload segment... Jul 18 02:20:00.553530248 (593h9m35.324171583s) [seg_pos={0 0}, upload=474403]: Done begin upload segment. Jul 18 02:20:00.553546579 (593h9m35.324187914s) [noise=true , address=45.91.95.2:4036 , piece_id=PQ4KHGVX6LPDS72E4EVVNIEK6BZHCCWBQ5NRKY3Y2EVYZHSP4XXA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.553575855 (593h9m35.32421719s) [noise=true , address=79.127.205.235:4664 , piece_id=OIKL32JRMNZ6ZOMSK46ECTKHFJGHDYDYTJJR7PCTIBTWRULZD7YA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.553932808 (593h9m35.324574143s) [noise=true , address=45.91.95.5:4011 , piece_id=XX26TUTMH6YQQJYCWAFQNEJ46DTI67PIQWUIVOTFDHXJB74ZPTQQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.554240527 (593h9m35.324881862s) [noise=true , address=104.36.82.49:4990 , piece_id=UDZIQXKCQKE4EXN2HE3WWADTKBPFVP45XNJELT3Z7GTVHNPCX42A, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.554395624 (593h9m35.325036959s) [noise=true , address=45.91.116.12:4253 , piece_id=HHDYLFB3PNQECC7DUMKDKYBFBCBPG3C7V42YRGF5VKX6BP2FIVXQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.554644941 (593h9m35.325286276s) [noise=true , address=104.36.82.45:4919 , piece_id=SUGUMHNWGMGEYHERHHPH5DI6E7DF6GRGTU2CABGFGWZTP5Y7UPKQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.554713922 (593h9m35.325355257s) [noise=true , address=104.36.82.13:4220 , piece_id=QCHNAQCOZRC7IICTFKVELBCYMKKPFVJ7CLUTE7SUD65B322KC4JA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.555000892 (593h9m35.325642227s) [noise=true , address=45.91.116.5:4314 , piece_id=EGBOTZXIC5C6SBONLD5MXPVHCTAQOB36BY56SVBX236AU7HZZIMA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.555006382 (593h9m35.325647717s) [noise=true , address=77.90.31.21:4886 , piece_id=ZKVRTNHX6TXMXL7ALSBJV3FM7WEATFZLKSM7NJ6IJSZ7DN3ZOBHQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.555312718 (593h9m35.325954053s) [noise=true , address=77.90.31.12:4622 , piece_id=24PKTTFNPNV5LIOH7NMIFC4Y4NZVGDLBNONWDZQ7YSAGT6VJF3HQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.555640616 (593h9m35.326281951s) [noise=true , address=77.90.31.13:4644 , piece_id=E7P3JDIU3DGBE2PFQO3KYBXATBYRJFWXY2JUMRG6O6L6BYGDOBPQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.555959196 (593h9m35.326600531s) [noise=true , address=45.91.95.3:4016 , piece_id=MOXXNTD6CXIDTLDONPTGYFV3LXOPPT3GGEVIYOZC4FGIFN4Y2UTA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.556323843 (593h9m35.326965178s) [noise=true , address=77.90.31.15:4700 , piece_id=J3S4PRXA4ED7KCVOBQTX37IPJXSSBK6NGHAWCW3PMU2TC7S47Q7A, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.556618988 (593h9m35.327260323s) [noise=true , address=77.90.31.11:4572 , piece_id=PN22DARG6NAN4HTT5VW4T33Y4GKO2O4EW7SAMBUBHITQH5SEEIFA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.557035845 (593h9m35.32767718s) [noise=true , address=77.90.31.18:4839 , piece_id=6WSFLBWA7PWNH3SIH6GTGJIPQZVWBBPECIO4IAT7NRFCD7XVRDGQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.557749401 (593h9m35.328390736s) [noise=true , address=45.91.116.6:4996 , piece_id=ZJDN2QL3NKZEG7ADXVK3IELTAVZOUG3CSMOOQ62ZEW5OZHQE6QJA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.560343844 (593h9m35.330985179s) [noise=true , address=45.91.116.7:4028 , piece_id=R6G22SFW6CRM6HUZXOGD2KMXMJ6CHJ2JERIH6O3MIEYI4W4DXKHA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.560643678 (593h9m35.331285013s) [noise=true , address=79.127.220.97:4363 , piece_id=LQOOSSB3PZVLGUJHJ3FESFYFU7EZM6SSZVWHZJMRARLQ2VZOOPUA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:01.660842919 (593h9m36.431484255s) [upload=474403]: Next returned nil segment Jul 18 02:20:01.660862567 (593h9m36.431503902s) [upload=474403]: Waiting for error group managing segments... Jul 18 02:20:02.086245288 (593h9m36.856886623s) [noise=true , address=79.127.205.235:4664 , piece_id=OIKL32JRMNZ6ZOMSK46ECTKHFJGHDYDYTJJR7PCTIBTWRULZD7YA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.231798885 (593h9m37.00244022s) [noise=true , address=79.127.220.97:4363 , piece_id=LQOOSSB3PZVLGUJHJ3FESFYFU7EZM6SSZVWHZJMRARLQ2VZOOPUA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.297293685 (593h9m37.06793502s) [noise=true , address=45.91.95.2:4036 , piece_id=PQ4KHGVX6LPDS72E4EVVNIEK6BZHCCWBQ5NRKY3Y2EVYZHSP4XXA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.420070793 (593h9m37.190712128s) [noise=true , address=45.91.95.5:4011 , piece_id=XX26TUTMH6YQQJYCWAFQNEJ46DTI67PIQWUIVOTFDHXJB74ZPTQQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.503188251 (593h9m37.273829596s) [noise=true , address=45.91.95.3:4016 , piece_id=MOXXNTD6CXIDTLDONPTGYFV3LXOPPT3GGEVIYOZC4FGIFN4Y2UTA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.509620257 (593h9m37.280261602s) [noise=true , address=45.91.116.5:4314 , piece_id=EGBOTZXIC5C6SBONLD5MXPVHCTAQOB36BY56SVBX236AU7HZZIMA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.511608351 (593h9m37.282249736s) [noise=true , address=104.36.82.13:4220 , piece_id=QCHNAQCOZRC7IICTFKVELBCYMKKPFVJ7CLUTE7SUD65B322KC4JA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.550968919 (593h9m37.321610264s) [noise=true , address=45.91.116.12:4253 , piece_id=HHDYLFB3PNQECC7DUMKDKYBFBCBPG3C7V42YRGF5VKX6BP2FIVXQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.556962767 (593h9m37.327604112s) [noise=true , address=45.91.116.7:4028 , piece_id=R6G22SFW6CRM6HUZXOGD2KMXMJ6CHJ2JERIH6O3MIEYI4W4DXKHA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.597868492 (593h9m37.368509827s) [noise=true , address=45.91.116.6:4996 , piece_id=ZJDN2QL3NKZEG7ADXVK3IELTAVZOUG3CSMOOQ62ZEW5OZHQE6QJA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.600015169 (593h9m37.370656514s) [noise=true , address=104.36.82.49:4990 , piece_id=UDZIQXKCQKE4EXN2HE3WWADTKBPFVP45XNJELT3Z7GTVHNPCX42A, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.661038495 (593h9m37.43167983s) [noise=true , address=104.36.82.45:4919 , piece_id=SUGUMHNWGMGEYHERHHPH5DI6E7DF6GRGTU2CABGFGWZTP5Y7UPKQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:59.365096061 (593h10m34.135737396s) [noise=true , address=77.90.31.12:4622 , piece_id=24PKTTFNPNV5LIOH7NMIFC4Y4NZVGDLBNONWDZQ7YSAGT6VJF3HQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: ecclient: upload failed (node:12MQryCwJkVN1JhFm4QqhypX3WoMDR9rdjELj4oYoVbpa6U823Z, address:77.90.31.12:4622): protocol: rpc: dial tcp 77.90.31.12:4622: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:59.365138882 (593h10m34.135780207s) [noise=true , address=77.90.31.13:4644 , piece_id=E7P3JDIU3DGBE2PFQO3KYBXATBYRJFWXY2JUMRG6O6L6BYGDOBPQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: ecclient: upload failed (node:13QCH91kRMQjVj2ASsVDcyXQ1aapQRVeUm6MMCGimsR6Ns973h, address:77.90.31.13:4644): protocol: rpc: dial tcp 77.90.31.13:4644: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:59.365168008 (593h10m34.135809343s) [noise=true , address=77.90.31.15:4700 , piece_id=J3S4PRXA4ED7KCVOBQTX37IPJXSSBK6NGHAWCW3PMU2TC7S47Q7A, seg_pos={0 0}, upload=474403]: Done uploading piece. err: ecclient: upload failed (node:124p2J9pP2rYD3cM73PQNEp6Ykgmpq2xi6aK8wnC6JTiLvHRb7q, address:77.90.31.15:4700): protocol: rpc: dial tcp 77.90.31.15:4700: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:59.365199549 (593h10m34.135840884s) [noise=true , address=77.90.31.11:4572 , piece_id=PN22DARG6NAN4HTT5VW4T33Y4GKO2O4EW7SAMBUBHITQH5SEEIFA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: ecclient: upload failed (node:1ZAfYZ4wgpZVvatj6wtHona3hTaPcLxAmXC1DT2uKQyo8nLQQ7, address:77.90.31.11:4572): protocol: rpc: dial tcp 77.90.31.11:4572: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:59.365230558 (593h10m34.135871893s) [noise=true , address=77.90.31.18:4839 , piece_id=6WSFLBWA7PWNH3SIH6GTGJIPQZVWBBPECIO4IAT7NRFCD7XVRDGQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: ecclient: upload failed (node:1gmWwyaQ6zM6QdayBFpXdE9VSE1aaQcBHFcukJoASjjHWCDsNx, address:77.90.31.18:4839): protocol: write tcp 188.166.225.170:41944->77.90.31.18:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:59.365265114 (593h10m34.135906449s) [noise=true , address=77.90.31.21:4886 , piece_id=ZKVRTNHX6TXMXL7ALSBJV3FM7WEATFZLKSM7NJ6IJSZ7DN3ZOBHQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: ecclient: upload failed (node:12VMSq3AX3PYjchs8n7yDWVWdKVRUVoe3PiyDitkX1EwSUwsTeM, address:77.90.31.21:4886): protocol: write tcp 188.166.225.170:48778->77.90.31.21:4886: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:59.365278149 (593h10m34.135919484s) [seg_pos={0 0}, upload=474403]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 18 02:20:59.618295247 (593h10m34.388936582s) [upload=474403]: Done uploading segments... Jul 18 02:20:59.618319955 (593h10m34.38896129s) [upload=474403]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 5.877971491s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.................................................X.X......XX] duration: 1m0.254585582s FAIL (2025-07-18 02:20:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== egin Jul 18 02:20:00.064079005 (593h9m34.83472034s) [upload=474403]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:00.553474341 (593h9m35.324115676s) [seg_pos={0 0}, upload=474403]: Begin upload segment... Jul 18 02:20:00.553530248 (593h9m35.324171583s) [seg_pos={0 0}, upload=474403]: Done begin upload segment. Jul 18 02:20:00.553546579 (593h9m35.324187914s) [noise=true , address=45.91.95.2:4036 , piece_id=PQ4KHGVX6LPDS72E4EVVNIEK6BZHCCWBQ5NRKY3Y2EVYZHSP4XXA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.553575855 (593h9m35.32421719s) [noise=true , address=79.127.205.235:4664 , piece_id=OIKL32JRMNZ6ZOMSK46ECTKHFJGHDYDYTJJR7PCTIBTWRULZD7YA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.553932808 (593h9m35.324574143s) [noise=true , address=45.91.95.5:4011 , piece_id=XX26TUTMH6YQQJYCWAFQNEJ46DTI67PIQWUIVOTFDHXJB74ZPTQQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.554240527 (593h9m35.324881862s) [noise=true , address=104.36.82.49:4990 , piece_id=UDZIQXKCQKE4EXN2HE3WWADTKBPFVP45XNJELT3Z7GTVHNPCX42A, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.554395624 (593h9m35.325036959s) [noise=true , address=45.91.116.12:4253 , piece_id=HHDYLFB3PNQECC7DUMKDKYBFBCBPG3C7V42YRGF5VKX6BP2FIVXQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.554644941 (593h9m35.325286276s) [noise=true , address=104.36.82.45:4919 , piece_id=SUGUMHNWGMGEYHERHHPH5DI6E7DF6GRGTU2CABGFGWZTP5Y7UPKQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.554713922 (593h9m35.325355257s) [noise=true , address=104.36.82.13:4220 , piece_id=QCHNAQCOZRC7IICTFKVELBCYMKKPFVJ7CLUTE7SUD65B322KC4JA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.555000892 (593h9m35.325642227s) [noise=true , address=45.91.116.5:4314 , piece_id=EGBOTZXIC5C6SBONLD5MXPVHCTAQOB36BY56SVBX236AU7HZZIMA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.555006382 (593h9m35.325647717s) [noise=true , address=77.90.31.21:4886 , piece_id=ZKVRTNHX6TXMXL7ALSBJV3FM7WEATFZLKSM7NJ6IJSZ7DN3ZOBHQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.555312718 (593h9m35.325954053s) [noise=true , address=77.90.31.12:4622 , piece_id=24PKTTFNPNV5LIOH7NMIFC4Y4NZVGDLBNONWDZQ7YSAGT6VJF3HQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.555640616 (593h9m35.326281951s) [noise=true , address=77.90.31.13:4644 , piece_id=E7P3JDIU3DGBE2PFQO3KYBXATBYRJFWXY2JUMRG6O6L6BYGDOBPQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.555959196 (593h9m35.326600531s) [noise=true , address=45.91.95.3:4016 , piece_id=MOXXNTD6CXIDTLDONPTGYFV3LXOPPT3GGEVIYOZC4FGIFN4Y2UTA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.556323843 (593h9m35.326965178s) [noise=true , address=77.90.31.15:4700 , piece_id=J3S4PRXA4ED7KCVOBQTX37IPJXSSBK6NGHAWCW3PMU2TC7S47Q7A, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.556618988 (593h9m35.327260323s) [noise=true , address=77.90.31.11:4572 , piece_id=PN22DARG6NAN4HTT5VW4T33Y4GKO2O4EW7SAMBUBHITQH5SEEIFA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.557035845 (593h9m35.32767718s) [noise=true , address=77.90.31.18:4839 , piece_id=6WSFLBWA7PWNH3SIH6GTGJIPQZVWBBPECIO4IAT7NRFCD7XVRDGQ, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.557749401 (593h9m35.328390736s) [noise=true , address=45.91.116.6:4996 , piece_id=ZJDN2QL3NKZEG7ADXVK3IELTAVZOUG3CSMOOQ62ZEW5OZHQE6QJA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.560343844 (593h9m35.330985179s) [noise=true , address=45.91.116.7:4028 , piece_id=R6G22SFW6CRM6HUZXOGD2KMXMJ6CHJ2JERIH6O3MIEYI4W4DXKHA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:00.560643678 (593h9m35.331285013s) [noise=true , address=79.127.220.97:4363 , piece_id=LQOOSSB3PZVLGUJHJ3FESFYFU7EZM6SSZVWHZJMRARLQ2VZOOPUA, seg_pos={0 0}, upload=474403]: Uploading piece... Jul 18 02:20:01.660842919 (593h9m36.431484255s) [upload=474403]: Next returned nil segment Jul 18 02:20:01.660862567 (593h9m36.431503902s) [upload=474403]: Waiting for error group managing segments... Jul 18 02:20:02.086245288 (593h9m36.856886623s) [noise=true , address=79.127.205.235:4664 , piece_id=OIKL32JRMNZ6ZOMSK46ECTKHFJGHDYDYTJJR7PCTIBTWRULZD7YA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.231798885 (593h9m37.00244022s) [noise=true , address=79.127.220.97:4363 , piece_id=LQOOSSB3PZVLGUJHJ3FESFYFU7EZM6SSZVWHZJMRARLQ2VZOOPUA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.297293685 (593h9m37.06793502s) [noise=true , address=45.91.95.2:4036 , piece_id=PQ4KHGVX6LPDS72E4EVVNIEK6BZHCCWBQ5NRKY3Y2EVYZHSP4XXA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.420070793 (593h9m37.190712128s) [noise=true , address=45.91.95.5:4011 , piece_id=XX26TUTMH6YQQJYCWAFQNEJ46DTI67PIQWUIVOTFDHXJB74ZPTQQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.503188251 (593h9m37.273829596s) [noise=true , address=45.91.95.3:4016 , piece_id=MOXXNTD6CXIDTLDONPTGYFV3LXOPPT3GGEVIYOZC4FGIFN4Y2UTA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.509620257 (593h9m37.280261602s) [noise=true , address=45.91.116.5:4314 , piece_id=EGBOTZXIC5C6SBONLD5MXPVHCTAQOB36BY56SVBX236AU7HZZIMA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.511608351 (593h9m37.282249736s) [noise=true , address=104.36.82.13:4220 , piece_id=QCHNAQCOZRC7IICTFKVELBCYMKKPFVJ7CLUTE7SUD65B322KC4JA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.550968919 (593h9m37.321610264s) [noise=true , address=45.91.116.12:4253 , piece_id=HHDYLFB3PNQECC7DUMKDKYBFBCBPG3C7V42YRGF5VKX6BP2FIVXQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.556962767 (593h9m37.327604112s) [noise=true , address=45.91.116.7:4028 , piece_id=R6G22SFW6CRM6HUZXOGD2KMXMJ6CHJ2JERIH6O3MIEYI4W4DXKHA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.597868492 (593h9m37.368509827s) [noise=true , address=45.91.116.6:4996 , piece_id=ZJDN2QL3NKZEG7ADXVK3IELTAVZOUG3CSMOOQ62ZEW5OZHQE6QJA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.600015169 (593h9m37.370656514s) [noise=true , address=104.36.82.49:4990 , piece_id=UDZIQXKCQKE4EXN2HE3WWADTKBPFVP45XNJELT3Z7GTVHNPCX42A, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:02.661038495 (593h9m37.43167983s) [noise=true , address=104.36.82.45:4919 , piece_id=SUGUMHNWGMGEYHERHHPH5DI6E7DF6GRGTU2CABGFGWZTP5Y7UPKQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: <nil> Jul 18 02:20:59.365096061 (593h10m34.135737396s) [noise=true , address=77.90.31.12:4622 , piece_id=24PKTTFNPNV5LIOH7NMIFC4Y4NZVGDLBNONWDZQ7YSAGT6VJF3HQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: ecclient: upload failed (node:12MQryCwJkVN1JhFm4QqhypX3WoMDR9rdjELj4oYoVbpa6U823Z, address:77.90.31.12:4622): protocol: rpc: dial tcp 77.90.31.12:4622: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:59.365138882 (593h10m34.135780207s) [noise=true , address=77.90.31.13:4644 , piece_id=E7P3JDIU3DGBE2PFQO3KYBXATBYRJFWXY2JUMRG6O6L6BYGDOBPQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: ecclient: upload failed (node:13QCH91kRMQjVj2ASsVDcyXQ1aapQRVeUm6MMCGimsR6Ns973h, address:77.90.31.13:4644): protocol: rpc: dial tcp 77.90.31.13:4644: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:59.365168008 (593h10m34.135809343s) [noise=true , address=77.90.31.15:4700 , piece_id=J3S4PRXA4ED7KCVOBQTX37IPJXSSBK6NGHAWCW3PMU2TC7S47Q7A, seg_pos={0 0}, upload=474403]: Done uploading piece. err: ecclient: upload failed (node:124p2J9pP2rYD3cM73PQNEp6Ykgmpq2xi6aK8wnC6JTiLvHRb7q, address:77.90.31.15:4700): protocol: rpc: dial tcp 77.90.31.15:4700: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:59.365199549 (593h10m34.135840884s) [noise=true , address=77.90.31.11:4572 , piece_id=PN22DARG6NAN4HTT5VW4T33Y4GKO2O4EW7SAMBUBHITQH5SEEIFA, seg_pos={0 0}, upload=474403]: Done uploading piece. err: ecclient: upload failed (node:1ZAfYZ4wgpZVvatj6wtHona3hTaPcLxAmXC1DT2uKQyo8nLQQ7, address:77.90.31.11:4572): protocol: rpc: dial tcp 77.90.31.11:4572: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:59.365230558 (593h10m34.135871893s) [noise=true , address=77.90.31.18:4839 , piece_id=6WSFLBWA7PWNH3SIH6GTGJIPQZVWBBPECIO4IAT7NRFCD7XVRDGQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: ecclient: upload failed (node:1gmWwyaQ6zM6QdayBFpXdE9VSE1aaQcBHFcukJoASjjHWCDsNx, address:77.90.31.18:4839): protocol: write tcp 188.166.225.170:41944->77.90.31.18:4839: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:59.365265114 (593h10m34.135906449s) [noise=true , address=77.90.31.21:4886 , piece_id=ZKVRTNHX6TXMXL7ALSBJV3FM7WEATFZLKSM7NJ6IJSZ7DN3ZOBHQ, seg_pos={0 0}, upload=474403]: Done uploading piece. err: ecclient: upload failed (node:12VMSq3AX3PYjchs8n7yDWVWdKVRUVoe3PiyDitkX1EwSUwsTeM, address:77.90.31.21:4886): protocol: write tcp 188.166.225.170:48778->77.90.31.21:4886: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:59.365278149 (593h10m34.135919484s) [seg_pos={0 0}, upload=474403]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 18 02:20:59.618295247 (593h10m34.388936582s) [upload=474403]: Done uploading segments... Jul 18 02:20:59.618319955 (593h10m34.38896129s) [upload=474403]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 1m0.254599228s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...............................................XX.......XXXX] duration: 30.251923272s FAIL (2025-07-18 02:20:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== 3388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 18 02:20:24.779878619 (593h9m59.550519954s) [upload=474407]: Starting upload Jul 18 02:20:24.779962249 (593h9m59.550603584s) [upload=474407]: Done starting upload Jul 18 02:20:24.780003679 (593h9m59.550645024s) [upload=474407]: Uploading segments... Jul 18 02:20:24.780157943 (593h9m59.550799278s) [upload=474407]: Got next segment. Inline: false Jul 18 02:20:24.780266601 (593h9m59.550907976s) [upload=474407]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:20:24.780275469 (593h9m59.550916824s) [upload=474407]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:20:25.031967328 (593h9m59.802608663s) [seg_pos={0 0}, upload=474407]: Begin upload segment... Jul 18 02:20:25.032006182 (593h9m59.802647517s) [seg_pos={0 0}, upload=474407]: Done begin upload segment. Jul 18 02:20:25.032029236 (593h9m59.802670572s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032034727 (593h9m59.802676062s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032066218 (593h9m59.802707543s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032490629 (593h9m59.803131965s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032679040 (593h9m59.803320375s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032750757 (593h9m59.803392092s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.032986588 (593h9m59.803627913s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033008651 (593h9m59.803649976s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033262296 (593h9m59.803903621s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033295740 (593h9m59.803937075s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033520239 (593h9m59.804161574s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033734439 (593h9m59.804375774s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.033953097 (593h9m59.804594432s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034150214 (593h9m59.804791549s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034350338 (593h9m59.804991663s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:25.034544279 (593h9m59.805185604s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Uploading piece... Jul 18 02:20:26.722719428 (593h10m1.493360753s) [upload=474407]: Next returned nil segment Jul 18 02:20:26.722737463 (593h10m1.493378798s) [upload=474407]: Waiting for error group managing segments... Jul 18 02:20:27.069826685 (593h10m1.84046802s) [noise=true , address=45.91.116.5:4298 , piece_id=BX4I6LTLCV36JBDKSG5HRW3KCI5RK5GZSYLNF6UF7JCQV62YSBFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.092893549 (593h10m1.863534884s) [noise=true , address=104.36.82.4:4804 , piece_id=L7QQQYBFIUZ5EAONATMIK4GX4V4SYLTFHJQLKWSXXFCU3JT7E7AQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.143535552 (593h10m1.914176887s) [noise=true , address=45.91.116.6:4973 , piece_id=GNALGJMD5QZS4UAO3VKUIIMNRUT5E5CFYFCNHFCMW4I53C43GQ5Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.151311830 (593h10m1.921953165s) [noise=true , address=45.91.116.4:4938 , piece_id=L7KPVBMUZNY5RZ67SGQRZAE4OPQCTD65A2HWCN6EHQ6NW5TRACFA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.154544595 (593h10m1.92518593s) [noise=true , address=45.91.116.8:4048 , piece_id=LWZWKGS2IIPE4TOXZIHWF5LTW3G5R6LL4EYNOBLG3LUGX76PZRDQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.167648668 (593h10m1.938290003s) [noise=true , address=104.36.82.8:4970 , piece_id=QYLXIOE3XZQIWLLCKLYCG6Q5ZYYSBOECYWENAF5T5L5UU7EB7WWA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210335119 (593h10m1.980976454s) [noise=true , address=104.36.82.2:4731 , piece_id=NVBPEEVKXWUXXJ27Y4IFX4IO4ICGN4LXNELKUIE3ZVJX4VL2LD4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.210500045 (593h10m1.98114138s) [noise=true , address=45.91.116.5:4297 , piece_id=I3FBBP7NI4RH5VTKFUWB5L3EH7BERV2CT2DEENVTV3X4KCOFLM4Q, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.232506489 (593h10m2.003147824s) [noise=true , address=45.91.116.14:4189 , piece_id=HOFJMFCK3S2T3YBUPIPTCY227U4E7XQASACJOF464MP6NFJK56DQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.234721757 (593h10m2.005363092s) [noise=true , address=104.36.82.13:4186 , piece_id=S5H5OLN7B5YA6S2JKVDMCP5UI2C5RCNXUBGBFIHPGGYMFIS522SA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.238800071 (593h10m2.009441406s) [noise=true , address=104.36.82.6:4873 , piece_id=SH6NJFVS2IJ5QMLO6WJURRMSWJAXD46K7UFPGYUEZQ25LYTSODPA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:27.254974146 (593h10m2.025615481s) [noise=true , address=104.36.82.16:4293 , piece_id=PS52OGX2Q4G2AIMOFVN2UAKWC6XTWDJ5TGK6CTYALBSJLMRT3BRA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: <nil> Jul 18 02:20:54.085289022 (593h10m28.855930357s) [noise=true , address=77.90.31.18:4806 , piece_id=E5IYEE2FVEMMHHV7YVDWUFQDZMDRDKATHTMFBLIYBR344KI3FTYQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:53138->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085367181 (593h10m28.856008517s) [noise=true , address=77.90.31.10:4519 , piece_id=WEZBWBRN3GPIC6QPXNTM5LPSV6UKUXLDGWWG7HOXXBXOY7KEWYOQ, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:35864->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085427958 (593h10m28.856069293s) [noise=true , address=77.90.31.14:4661 , piece_id=TXO55SI4F2VW356XSS4M3QSN57WW7I2YV65BW3GCLAWKTUBAN44A, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085460200 (593h10m28.856101535s) [noise=true , address=77.90.31.17:4771 , piece_id=UXQMWPDAZP7UEHH4AEMOXBNLVZHMMX5VPMDVWHQWK24NXMVY7MYA, seg_pos={0 0}, upload=474407]: Done uploading piece. err: ecclient: upload failed (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2, address:77.90.31.17:4771): protocol: rpc: dial tcp 77.90.31.17:4771: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:54.085487322 (593h10m28.856128657s) [seg_pos={0 0}, upload=474407]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:20:54.335312937 (593h10m29.105954273s) [upload=474407]: Done uploading segments... Jul 18 02:20:54.335338926 (593h10m29.105980261s) [upload=474407]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.2519474s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [................................................XX.......XXX] duration: 30.251242334s FAIL (2025-07-18 02:20:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== :45.91.116.10:4083): protocol: write tcp 188.166.225.170:35874->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:33.492665055 (593h9m8.26330639s) [seg_pos={0 0}, upload=474390]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 18 02:19:33.736370133 (593h9m8.507011469s) [upload=474390]: Done uploading segments... Jul 18 02:19:33.736395702 (593h9m8.507037037s) [upload=474390]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Jul 18 02:19:34.656674644 (593h9m9.427315979s) [upload=474395]: Starting upload Jul 18 02:19:34.656744267 (593h9m9.427385603s) [upload=474395]: Done starting upload Jul 18 02:19:34.656834620 (593h9m9.427475945s) [upload=474395]: Uploading segments... Jul 18 02:19:34.656952876 (593h9m9.427594221s) [upload=474395]: Got next segment. Inline: false Jul 18 02:19:34.656984747 (593h9m9.427626082s) [upload=474395]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:19:34.656989717 (593h9m9.427631052s) [upload=474395]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:19:34.907796262 (593h9m9.678437597s) [seg_pos={0 0}, upload=474395]: Begin upload segment... Jul 18 02:19:34.907842671 (593h9m9.678483996s) [seg_pos={0 0}, upload=474395]: Done begin upload segment. Jul 18 02:19:34.907856277 (593h9m9.678497603s) [noise=true , address=45.91.116.14:4189 , piece_id=TB32VH7NAXPQFPSZL5ZXWCJK7S6JZ6KWDJ43GLJ2GDSGORL4G6XQ, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.907865365 (593h9m9.6785067s) [noise=true , address=77.90.31.16:4733 , piece_id=FDIDXIMVNOKTN5BFBYQIOPFIZLAQN2Z2W7G6R5JUJF5BTOOJCF4Q, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.908245813 (593h9m9.678887148s) [noise=true , address=45.91.116.4:4940 , piece_id=ID6URN4NJMZJBBKFKMK73YVSD2LXNGENZ6T6X6EJGOZTS2QSSXTQ, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.908254208 (593h9m9.678895543s) [noise=true , address=104.36.82.4:4803 , piece_id=HQOGL6KYGZNHRC6JC46BQERKUGGRVVXZMCHV27ZSHP74MLB3ZFDQ, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.908649064 (593h9m9.679290399s) [noise=true , address=104.36.82.8:4971 , piece_id=RNOAQ3GNTEEQDVXGV5TFDP5RX6Z4XEZGWPMRWZ33ZHBUG54TLMYA, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.909292966 (593h9m9.679934301s) [noise=true , address=104.36.82.7:4909 , piece_id=ZJONEW6ZWOVAAKKPTNJ2NGSEGS6O2QJUTKDNPF4QYHKV5JHIJBAA, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.909572170 (593h9m9.680213505s) [noise=true , address=104.36.82.10:4078 , piece_id=ESAQW6NYYOD54EVMYOBV4M6TTHAZ264PPYYMJR5MXJRYIHRMCEHQ, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.909879668 (593h9m9.680521003s) [noise=true , address=77.90.31.21:4879 , piece_id=CXSXZX6F6VD3FXRYVWFGWMQFA4HUFHIVV5FHR6YHZPCBIVPIJHIQ, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.910129616 (593h9m9.680770961s) [noise=true , address=104.36.82.11:4114 , piece_id=CRGPO2TAFF7Y46B2OPEA4AXTSCWNWITFGYDKG2OQQHLDQXYIQFYA, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.910326443 (593h9m9.680967778s) [noise=true , address=104.36.82.13:4185 , piece_id=HPH7ECBX45GQIGUAG25M54OY75F4SFYSMZMDSYLQ2C3PEVMLUNRQ, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.910529031 (593h9m9.681170356s) [noise=true , address=77.90.31.19:4841 , piece_id=ESFL5OJUTK5IOZPMSTPZ7HUYLGHVRE6RVT2D6Y4HINYN6TUDIIAQ, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.910738621 (593h9m9.681379946s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.910950547 (593h9m9.681591882s) [noise=true , address=45.91.116.13:4154 , piece_id=4IHYLRUF2V4RPXYIMDELCAKVI567KUH2IV5ILPCMZPPGYQF7VSZA, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.911144578 (593h9m9.681785913s) [noise=true , address=45.91.116.7:4010 , piece_id=OWEY6PRUPMOMXUIXWQZPXTDXFRC52HOPOAPYTYFAY3JUIM4OSXPA, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.911332308 (593h9m9.681973633s) [noise=true , address=77.90.31.12:4589 , piece_id=SEPIRIDAMJE4PELOEL75XVP6Q2UWJXOVGBMUUDL4P2U5AC2MUBWA, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:34.912120246 (593h9m9.682761581s) [noise=true , address=104.36.82.6:4873 , piece_id=GTBYYZS53XZHPUW7HK2Q6Q26SMYX23M46BRCW2BUVUMWYW2S552Q, seg_pos={0 0}, upload=474395]: Uploading piece... Jul 18 02:19:36.602377129 (593h9m11.373018464s) [upload=474395]: Next returned nil segment Jul 18 02:19:36.602393541 (593h9m11.373034876s) [upload=474395]: Waiting for error group managing segments... Jul 18 02:19:36.946459668 (593h9m11.717101003s) [noise=true , address=104.36.82.4:4803 , piece_id=HQOGL6KYGZNHRC6JC46BQERKUGGRVVXZMCHV27ZSHP74MLB3ZFDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: <nil> Jul 18 02:19:36.951415219 (593h9m11.722056544s) [noise=true , address=104.36.82.6:4873 , piece_id=GTBYYZS53XZHPUW7HK2Q6Q26SMYX23M46BRCW2BUVUMWYW2S552Q, seg_pos={0 0}, upload=474395]: Done uploading piece. err: <nil> Jul 18 02:19:36.957646813 (593h9m11.728288148s) [noise=true , address=104.36.82.13:4185 , piece_id=HPH7ECBX45GQIGUAG25M54OY75F4SFYSMZMDSYLQ2C3PEVMLUNRQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: <nil> Jul 18 02:19:36.967446231 (593h9m11.738087566s) [noise=true , address=104.36.82.11:4114 , piece_id=CRGPO2TAFF7Y46B2OPEA4AXTSCWNWITFGYDKG2OQQHLDQXYIQFYA, seg_pos={0 0}, upload=474395]: Done uploading piece. err: <nil> Jul 18 02:19:37.056828888 (593h9m11.827470233s) [noise=true , address=104.36.82.7:4909 , piece_id=ZJONEW6ZWOVAAKKPTNJ2NGSEGS6O2QJUTKDNPF4QYHKV5JHIJBAA, seg_pos={0 0}, upload=474395]: Done uploading piece. err: <nil> Jul 18 02:19:37.110920490 (593h9m11.881561815s) [noise=true , address=104.36.82.8:4971 , piece_id=RNOAQ3GNTEEQDVXGV5TFDP5RX6Z4XEZGWPMRWZ33ZHBUG54TLMYA, seg_pos={0 0}, upload=474395]: Done uploading piece. err: <nil> Jul 18 02:19:37.190760277 (593h9m11.961401612s) [noise=true , address=104.36.82.10:4078 , piece_id=ESAQW6NYYOD54EVMYOBV4M6TTHAZ264PPYYMJR5MXJRYIHRMCEHQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: <nil> Jul 18 02:19:52.617377962 (593h9m27.388019297s) [noise=true , address=45.91.116.4:4940 , piece_id=ID6URN4NJMZJBBKFKMK73YVSD2LXNGENZ6T6X6EJGOZTS2QSSXTQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: <nil> Jul 18 02:19:52.624199694 (593h9m27.394841029s) [noise=true , address=45.91.116.13:4154 , piece_id=4IHYLRUF2V4RPXYIMDELCAKVI567KUH2IV5ILPCMZPPGYQF7VSZA, seg_pos={0 0}, upload=474395]: Done uploading piece. err: <nil> Jul 18 02:19:52.675899090 (593h9m27.446540415s) [noise=true , address=45.91.116.7:4010 , piece_id=OWEY6PRUPMOMXUIXWQZPXTDXFRC52HOPOAPYTYFAY3JUIM4OSXPA, seg_pos={0 0}, upload=474395]: Done uploading piece. err: <nil> Jul 18 02:19:52.689504961 (593h9m27.460146296s) [noise=true , address=45.91.116.14:4189 , piece_id=TB32VH7NAXPQFPSZL5ZXWCJK7S6JZ6KWDJ43GLJ2GDSGORL4G6XQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: <nil> Jul 18 02:20:03.962130853 (593h9m38.732772198s) [noise=true , address=77.90.31.16:4733 , piece_id=FDIDXIMVNOKTN5BFBYQIOPFIZLAQN2Z2W7G6R5JUJF5BTOOJCF4Q, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: write tcp 188.166.225.170:45234->77.90.31.16:4733: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962263236 (593h9m38.732904581s) [noise=true , address=77.90.31.19:4841 , piece_id=ESFL5OJUTK5IOZPMSTPZ7HUYLGHVRE6RVT2D6Y4HINYN6TUDIIAQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:55034->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962396842 (593h9m38.733038187s) [noise=true , address=77.90.31.21:4879 , piece_id=CXSXZX6F6VD3FXRYVWFGWMQFA4HUFHIVV5FHR6YHZPCBIVPIJHIQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962585974 (593h9m38.733227319s) [noise=true , address=77.90.31.12:4589 , piece_id=SEPIRIDAMJE4PELOEL75XVP6Q2UWJXOVGBMUUDL4P2U5AC2MUBWA, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:33310->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962747233 (593h9m38.733388568s) [noise=true , address=77.90.31.11:4554 , piece_id=6AWUJQ2FYS6GDPCZKMEXY3RI3ZN5P5VZGL2ECHWNWKOMBBGS7NDQ, seg_pos={0 0}, upload=474395]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:51514->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:20:03.962877852 (593h9m38.733519217s) [seg_pos={0 0}, upload=474395]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 18 02:20:04.210127302 (593h9m38.980768647s) [upload=474395]: Done uploading segments... Jul 18 02:20:04.210159654 (593h9m38.980800989s) [upload=474395]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.25126705s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..................................................X.X......X] duration: 1m0.272226063s FAIL (2025-07-18 02:19:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== . Jul 18 02:19:00.044507044 (593h8m34.815148379s) [seg_pos={0 0}, upload=474389]: Done begin upload segment. Jul 18 02:19:00.044549254 (593h8m34.815190589s) [noise=true , address=77.90.31.17:4804 , piece_id=VU6GCY7YLP2GLSIKIVD5A2TL57HC6A2U3S4ZASVP2F5B5SZWZBXA, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.044567650 (593h8m34.815208985s) [noise=true , address=45.91.116.12:4254 , piece_id=MGSJZJGAEGZ6AION2J6XECF5YXR5YTYEPZIAW32ZQ7P6HMDEGCAQ, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.044644046 (593h8m34.815285381s) [noise=true , address=156.146.43.225:4462 , piece_id=5S6NZFIJLHDW3VRIIZMV3LPVIBFEJNORSURM3XHOBKZ2F2C6YXXQ, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.044970180 (593h8m34.815611515s) [noise=true , address=104.36.82.10:4086 , piece_id=VM4PGXUBYX5EH75AQYKF3PYSIF25TOVKZ7AQLYJNYY5HUGAYO26Q, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.045105058 (593h8m34.815746393s) [noise=true , address=77.90.31.18:4825 , piece_id=T2KX7QVJY565GZGCOUR7UXCOK6R3PABHHDA6SZZU2L4HBJFHLUZQ, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.045180321 (593h8m34.815821657s) [noise=true , address=45.91.95.2:4014 , piece_id=OQILNBMQLSMER4DKXKJZ3VTY47GCPYKIHRRRX42ILYJUIPMT4ILA, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.045505494 (593h8m34.816146829s) [noise=true , address=77.90.31.15:4700 , piece_id=ODRO2HZNCRIJ2VS3QS6YXBIQTOPVMWKRR55EC6T52A4YDVS4AGZQ, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.046004589 (593h8m34.816645924s) [noise=true , address=45.91.95.3:4017 , piece_id=Q6KUTTMHCL3FUYVG5Z7QSQMKNQBHAC2DHQBCDFBEOJQK3QVY76MQ, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.046326334 (593h8m34.81696767s) [noise=true , address=45.91.116.6:4991 , piece_id=2GR2YPYX3EGIHKCJJCRBMH6JS5TRHZPTXOUXHPIG3I4MXQN57KSA, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.046683638 (593h8m34.817324983s) [noise=true , address=104.36.82.2:4739 , piece_id=PRMHGJWSP4HDFRAXSXFGV3UG2CBQPQ4C3G5EHGYVS7FM4BUDKBXQ, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.047063806 (593h8m34.817705141s) [noise=true , address=79.127.205.234:4618 , piece_id=TD6H3LJ72EIT3ZKUCM5UBNBQPIWXAGEELYLDKXI5B6LE4OH733NQ, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.047404126 (593h8m34.818045461s) [noise=true , address=79.127.219.38:4022 , piece_id=VOJV4DONXE37HEPCG2DXVT7QIT2O45VMU4EVEMJWJS63VKLTFXJQ, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.047725742 (593h8m34.818367077s) [noise=true , address=45.91.116.13:4156 , piece_id=UA2OSYMHN4RPA7HBEXLSPUIDUO2CPLXZYCRTOLCGTA65LMK6JTQQ, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.048057747 (593h8m34.818699082s) [noise=true , address=104.36.82.12:4167 , piece_id=NADPR2USVJOGPLBF3SJEH7BRPVEGKAKR74EFQIG6XDDQYJXLYE2Q, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.048441612 (593h8m34.819082947s) [noise=true , address=45.91.95.5:4034 , piece_id=VJHTB3HAVEY4KW5BIMOTIPHBGQHLN3OFZ3HRWAH5TVPP6KGEYRWA, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.048788937 (593h8m34.819430272s) [noise=true , address=45.91.116.14:4200 , piece_id=I2Y5DH4NUJJBJBZKMW3XO7GB3AQPKWIJTR5XLD7UI4KXJGNDBS5Q, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.051243533 (593h8m34.821884868s) [noise=true , address=104.36.82.45:4915 , piece_id=EIJ5ZCPT4VSJWIXONGBLZTJHVHTNOOFVJHKDP4D3LSM2OJGCBBRA, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:00.053319875 (593h8m34.82396121s) [noise=true , address=45.91.116.11:4145 , piece_id=C6XJPVTEWKMS3BZLQOUGCTJ4OJOB4AS2X3XUKSLZP7YWNGMMC3JA, seg_pos={0 0}, upload=474389]: Uploading piece... Jul 18 02:19:01.162577778 (593h8m35.933219113s) [upload=474389]: Next returned nil segment Jul 18 02:19:01.162595212 (593h8m35.933236547s) [upload=474389]: Waiting for error group managing segments... Jul 18 02:19:01.780150023 (593h8m36.550791359s) [noise=true , address=45.91.95.3:4017 , piece_id=Q6KUTTMHCL3FUYVG5Z7QSQMKNQBHAC2DHQBCDFBEOJQK3QVY76MQ, seg_pos={0 0}, upload=474389]: Done uploading piece. err: <nil> Jul 18 02:19:01.822374781 (593h8m36.593016117s) [noise=true , address=45.91.95.2:4014 , piece_id=OQILNBMQLSMER4DKXKJZ3VTY47GCPYKIHRRRX42ILYJUIPMT4ILA, seg_pos={0 0}, upload=474389]: Done uploading piece. err: <nil> Jul 18 02:19:01.947492077 (593h8m36.718133422s) [noise=true , address=156.146.43.225:4462 , piece_id=5S6NZFIJLHDW3VRIIZMV3LPVIBFEJNORSURM3XHOBKZ2F2C6YXXQ, seg_pos={0 0}, upload=474389]: Done uploading piece. err: <nil> Jul 18 02:19:01.972985413 (593h8m36.743626748s) [noise=true , address=104.36.82.45:4915 , piece_id=EIJ5ZCPT4VSJWIXONGBLZTJHVHTNOOFVJHKDP4D3LSM2OJGCBBRA, seg_pos={0 0}, upload=474389]: Done uploading piece. err: <nil> Jul 18 02:19:01.975670751 (593h8m36.746312087s) [noise=true , address=79.127.219.38:4022 , piece_id=VOJV4DONXE37HEPCG2DXVT7QIT2O45VMU4EVEMJWJS63VKLTFXJQ, seg_pos={0 0}, upload=474389]: Done uploading piece. err: <nil> Jul 18 02:19:02.003082088 (593h8m36.773723423s) [noise=true , address=79.127.205.234:4618 , piece_id=TD6H3LJ72EIT3ZKUCM5UBNBQPIWXAGEELYLDKXI5B6LE4OH733NQ, seg_pos={0 0}, upload=474389]: Done uploading piece. err: <nil> Jul 18 02:19:02.019058655 (593h8m36.78969999s) [noise=true , address=104.36.82.10:4086 , piece_id=VM4PGXUBYX5EH75AQYKF3PYSIF25TOVKZ7AQLYJNYY5HUGAYO26Q, seg_pos={0 0}, upload=474389]: Done uploading piece. err: <nil> Jul 18 02:19:02.050116779 (593h8m36.820758114s) [noise=true , address=104.36.82.12:4167 , piece_id=NADPR2USVJOGPLBF3SJEH7BRPVEGKAKR74EFQIG6XDDQYJXLYE2Q, seg_pos={0 0}, upload=474389]: Done uploading piece. err: <nil> Jul 18 02:19:02.054774299 (593h8m36.825415634s) [noise=true , address=104.36.82.2:4739 , piece_id=PRMHGJWSP4HDFRAXSXFGV3UG2CBQPQ4C3G5EHGYVS7FM4BUDKBXQ, seg_pos={0 0}, upload=474389]: Done uploading piece. err: <nil> Jul 18 02:19:02.104961312 (593h8m36.875602657s) [noise=true , address=45.91.95.5:4034 , piece_id=VJHTB3HAVEY4KW5BIMOTIPHBGQHLN3OFZ3HRWAH5TVPP6KGEYRWA, seg_pos={0 0}, upload=474389]: Done uploading piece. err: <nil> Jul 18 02:19:58.862080533 (593h9m33.632721878s) [noise=true , address=45.91.116.14:4200 , piece_id=I2Y5DH4NUJJBJBZKMW3XO7GB3AQPKWIJTR5XLD7UI4KXJGNDBS5Q, seg_pos={0 0}, upload=474389]: Done uploading piece. err: ecclient: upload failed (node:12fiQJkHYDv2J4V82doFX3dg43hLpgq3c34Eng3zwqFkF5znyx, address:45.91.116.14:4200): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:19:58.862111051 (593h9m33.632752386s) [noise=true , address=45.91.116.11:4145 , piece_id=C6XJPVTEWKMS3BZLQOUGCTJ4OJOB4AS2X3XUKSLZP7YWNGMMC3JA, seg_pos={0 0}, upload=474389]: Done uploading piece. err: ecclient: upload failed (node:1WKV8PJxq5Kyb3ubCbrfbyYaX6fvRkPMKW2ssUteMFNHm9fP4E, address:45.91.116.11:4145): protocol: rpc: dial tcp 45.91.116.11:4145: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:58.862182698 (593h9m33.632824033s) [noise=true , address=45.91.116.6:4991 , piece_id=2GR2YPYX3EGIHKCJJCRBMH6JS5TRHZPTXOUXHPIG3I4MXQN57KSA, seg_pos={0 0}, upload=474389]: Done uploading piece. err: ecclient: upload failed (node:12muT2wQRGoRoDtpsPGS3EDjQ9sLq2qTS3jRSHWfuV1h9itzMbb, address:45.91.116.6:4991): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:19:58.862263052 (593h9m33.632904387s) [noise=true , address=45.91.116.13:4156 , piece_id=UA2OSYMHN4RPA7HBEXLSPUIDUO2CPLXZYCRTOLCGTA65LMK6JTQQ, seg_pos={0 0}, upload=474389]: Done uploading piece. err: ecclient: upload failed (node:1uZ5svEKyXSyB2kA3DjkHS976ioUcNjFehKCcxQY2pi9iEaAAy, address:45.91.116.13:4156): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:19:58.862331893 (593h9m33.632973239s) [noise=true , address=77.90.31.18:4825 , piece_id=T2KX7QVJY565GZGCOUR7UXCOK6R3PABHHDA6SZZU2L4HBJFHLUZQ, seg_pos={0 0}, upload=474389]: Done uploading piece. err: ecclient: upload failed (node:12PqKyfFxfVKTGARbPqBcQL3Naqjdf5uR44PeLi3sJpu57dcm1x, address:77.90.31.18:4825): protocol: rpc: dial tcp 77.90.31.18:4825: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:58.862341643 (593h9m33.632982978s) [noise=true , address=77.90.31.15:4700 , piece_id=ODRO2HZNCRIJ2VS3QS6YXBIQTOPVMWKRR55EC6T52A4YDVS4AGZQ, seg_pos={0 0}, upload=474389]: Done uploading piece. err: ecclient: upload failed (node:124p2J9pP2rYD3cM73PQNEp6Ykgmpq2xi6aK8wnC6JTiLvHRb7q, address:77.90.31.15:4700): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:19:58.862485047 (593h9m33.633126412s) [noise=true , address=45.91.116.12:4254 , piece_id=MGSJZJGAEGZ6AION2J6XECF5YXR5YTYEPZIAW32ZQ7P6HMDEGCAQ, seg_pos={0 0}, upload=474389]: Done uploading piece. err: ecclient: upload failed (node:12wAKmaLHwjBKN8UozGFjGvNMYG57fCxTH8wuDZDRnMdyGBRsTv, address:45.91.116.12:4254): protocol: write tcp 188.166.225.170:59084->45.91.116.12:4254: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:58.862513632 (593h9m33.633154967s) [noise=true , address=77.90.31.17:4804 , piece_id=VU6GCY7YLP2GLSIKIVD5A2TL57HC6A2U3S4ZASVP2F5B5SZWZBXA, seg_pos={0 0}, upload=474389]: Done uploading piece. err: ecclient: upload failed (node:168Tw7W69Tq9opDKhJ4cM8PGnzpxp9E9SxKbdVxJeF83pETqGx, address:77.90.31.17:4804): protocol: write tcp 188.166.225.170:55626->77.90.31.17:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:58.862533590 (593h9m33.633174925s) [seg_pos={0 0}, upload=474389]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 18 02:19:59.132001571 (593h9m33.902642896s) [upload=474389]: Done uploading segments... Jul 18 02:19:59.132028222 (593h9m33.902669557s) [upload=474389]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.272251382s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................................XX.......XX] duration: 30.245067066s FAIL (2025-07-18 02:19:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:19:04.474109995 (593h8m39.24475133s) [seg_pos={0 0}, upload=474390]: Begin upload segment... Jul 18 02:19:04.474163949 (593h8m39.244805284s) [seg_pos={0 0}, upload=474390]: Done begin upload segment. Jul 18 02:19:04.474185599 (593h8m39.244826934s) [noise=true , address=45.91.116.6:4975 , piece_id=LY275E3YLEV7FIF7S6L4IKXNHYVHDLUD7YYG6I7ZDHB6E6ZIP6DA, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.474227710 (593h8m39.244869045s) [noise=true , address=104.36.82.5:4840 , piece_id=BUK52B6LPUKND2DIHONX5HCCWK6EAL3KKV4O7BP6MSFJVSPHKHTA, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.474525190 (593h8m39.245166525s) [noise=true , address=104.36.82.7:4912 , piece_id=ITWYZ5SLYEWXV2OXG3AQZDWETFTJNU5NSRLCRB6UTLG72FYPNT4A, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.474619450 (593h8m39.245260785s) [noise=true , address=45.91.116.6:4973 , piece_id=EHZZSYVPJMDQDL656PL2DRN5CR55EK6ST5W66OVJ77WJSTTSSPOQ, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.474776370 (593h8m39.245417705s) [noise=true , address=104.36.82.10:4079 , piece_id=46C3DHCWDTLLPZTKKMJKV3FMUFTPOWDQ64T7MEVGCD4JPYNQZWNA, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.474881862 (593h8m39.245523197s) [noise=true , address=77.90.31.18:4805 , piece_id=ES47NVOWVS72ZBBCHREKXBOCIJDHU7BZ3X63E47JI5ATN5YBZHJA, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.475033742 (593h8m39.245675077s) [noise=true , address=45.91.116.11:4119 , piece_id=QHXC363JKCTH54KINTOREQSND7OR2GWINSYJCI3WP3QBZ5JEQKUQ, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.475117633 (593h8m39.245758968s) [noise=true , address=104.36.82.4:4804 , piece_id=TC6CD7DMRD2SJVDOJ7JLL5T6GHBN5YBYN56HIUUZ4IS27ZFDAKKQ, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.475288701 (593h8m39.245930036s) [noise=true , address=45.91.116.10:4083 , piece_id=4O2OO6LVNWMYASRYQAVO5VPFR4TPBQ3IIXZFZH56T6HXRCNVJQIA, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.475348906 (593h8m39.245990241s) [noise=true , address=104.36.82.15:4258 , piece_id=CF3PXTPKK25J4P6TT7VS653MWW45MXT3BI5VSB2NL4ONBPF5H72A, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.475527798 (593h8m39.246169133s) [noise=true , address=77.90.31.14:4662 , piece_id=I52JE46WBF5JTIPTFAWPAPSNCCFVFKBXHG4VLNBDJDKBCPY3DISQ, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.475590148 (593h8m39.246231483s) [noise=true , address=77.90.31.17:4772 , piece_id=76YJOQJ4IWFYFNVTX4S4OK6H544RLQVRP6C3RILGY7ZVZVQSXO5A, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.475723171 (593h8m39.246364506s) [noise=true , address=77.90.31.19:4842 , piece_id=5LZ2HS2HMTN25S6HOLVKMZR2QKEYXMBKGC4VH5S6QXNGCUK5WH7Q, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.475928224 (593h8m39.246569559s) [noise=true , address=104.36.82.6:4873 , piece_id=CDPQELOVVNTKD2X66JIWBE4SZJUBGXIQNQ4JCOMMBR65KRHLPQXA, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.476131463 (593h8m39.246772798s) [noise=true , address=104.36.82.5:4839 , piece_id=MNC643JQMBYZD4E7QZDKALLK2DQ25NF4TLO36VOGI27MKSQNGWTA, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:04.476420195 (593h8m39.247061531s) [noise=true , address=45.91.116.7:4010 , piece_id=PR4FHHKNOAXQKLMX32MOQPZW7WHLC5VG6EDLIS6AD5PQF7CVE2OA, seg_pos={0 0}, upload=474390]: Uploading piece... Jul 18 02:19:06.234386834 (593h8m41.005028169s) [upload=474390]: Next returned nil segment Jul 18 02:19:06.234421871 (593h8m41.005063206s) [upload=474390]: Waiting for error group managing segments... Jul 18 02:19:06.525471453 (593h8m41.296112788s) [noise=true , address=104.36.82.4:4804 , piece_id=TC6CD7DMRD2SJVDOJ7JLL5T6GHBN5YBYN56HIUUZ4IS27ZFDAKKQ, seg_pos={0 0}, upload=474390]: Done uploading piece. err: <nil> Jul 18 02:19:06.541579653 (593h8m41.312220988s) [noise=true , address=104.36.82.10:4079 , piece_id=46C3DHCWDTLLPZTKKMJKV3FMUFTPOWDQ64T7MEVGCD4JPYNQZWNA, seg_pos={0 0}, upload=474390]: Done uploading piece. err: <nil> Jul 18 02:19:06.612686701 (593h8m41.383328026s) [noise=true , address=104.36.82.7:4912 , piece_id=ITWYZ5SLYEWXV2OXG3AQZDWETFTJNU5NSRLCRB6UTLG72FYPNT4A, seg_pos={0 0}, upload=474390]: Done uploading piece. err: <nil> Jul 18 02:19:06.667168561 (593h8m41.437809886s) [noise=true , address=104.36.82.6:4873 , piece_id=CDPQELOVVNTKD2X66JIWBE4SZJUBGXIQNQ4JCOMMBR65KRHLPQXA, seg_pos={0 0}, upload=474390]: Done uploading piece. err: <nil> Jul 18 02:19:06.672285030 (593h8m41.442926365s) [noise=true , address=104.36.82.5:4840 , piece_id=BUK52B6LPUKND2DIHONX5HCCWK6EAL3KKV4O7BP6MSFJVSPHKHTA, seg_pos={0 0}, upload=474390]: Done uploading piece. err: <nil> Jul 18 02:19:06.673494524 (593h8m41.444135859s) [noise=true , address=104.36.82.5:4839 , piece_id=MNC643JQMBYZD4E7QZDKALLK2DQ25NF4TLO36VOGI27MKSQNGWTA, seg_pos={0 0}, upload=474390]: Done uploading piece. err: <nil> Jul 18 02:19:06.706877325 (593h8m41.47751866s) [noise=true , address=104.36.82.15:4258 , piece_id=CF3PXTPKK25J4P6TT7VS653MWW45MXT3BI5VSB2NL4ONBPF5H72A, seg_pos={0 0}, upload=474390]: Done uploading piece. err: <nil> Jul 18 02:19:33.492457918 (593h9m8.263099243s) [noise=true , address=77.90.31.18:4805 , piece_id=ES47NVOWVS72ZBBCHREKXBOCIJDHU7BZ3X63E47JI5ATN5YBZHJA, seg_pos={0 0}, upload=474390]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: write tcp 188.166.225.170:52100->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:33.492500019 (593h9m8.263141354s) [noise=true , address=77.90.31.17:4772 , piece_id=76YJOQJ4IWFYFNVTX4S4OK6H544RLQVRP6C3RILGY7ZVZVQSXO5A, seg_pos={0 0}, upload=474390]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:37464->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:33.492527612 (593h9m8.263168937s) [noise=true , address=45.91.116.6:4975 , piece_id=LY275E3YLEV7FIF7S6L4IKXNHYVHDLUD7YYG6I7ZDHB6E6ZIP6DA, seg_pos={0 0}, upload=474390]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: write tcp 188.166.225.170:38552->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:33.492547230 (593h9m8.263188565s) [noise=true , address=77.90.31.14:4662 , piece_id=I52JE46WBF5JTIPTFAWPAPSNCCFVFKBXHG4VLNBDJDKBCPY3DISQ, seg_pos={0 0}, upload=474390]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 188.166.225.170:57028->77.90.31.14:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:33.492575924 (593h9m8.26321726s) [noise=true , address=45.91.116.7:4010 , piece_id=PR4FHHKNOAXQKLMX32MOQPZW7WHLC5VG6EDLIS6AD5PQF7CVE2OA, seg_pos={0 0}, upload=474390]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: write tcp 188.166.225.170:49232->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:33.492596423 (593h9m8.263237758s) [noise=true , address=45.91.116.6:4973 , piece_id=EHZZSYVPJMDQDL656PL2DRN5CR55EK6ST5W66OVJ77WJSTTSSPOQ, seg_pos={0 0}, upload=474390]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:38540->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:33.492604048 (593h9m8.263245383s) [noise=true , address=45.91.116.11:4119 , piece_id=QHXC363JKCTH54KINTOREQSND7OR2GWINSYJCI3WP3QBZ5JEQKUQ, seg_pos={0 0}, upload=474390]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: write tcp 188.166.225.170:55018->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:33.492634266 (593h9m8.263275601s) [noise=true , address=77.90.31.19:4842 , piece_id=5LZ2HS2HMTN25S6HOLVKMZR2QKEYXMBKGC4VH5S6QXNGCUK5WH7Q, seg_pos={0 0}, upload=474390]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: write tcp 188.166.225.170:51478->77.90.31.19:4842: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:33.492645748 (593h9m8.263287083s) [noise=true , address=45.91.116.10:4083 , piece_id=4O2OO6LVNWMYASRYQAVO5VPFR4TPBQ3IIXZFZH56T6HXRCNVJQIA, seg_pos={0 0}, upload=474390]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 188.166.225.170:35874->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:33.492665055 (593h9m8.26330639s) [seg_pos={0 0}, upload=474390]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 18 02:19:33.736370133 (593h9m8.507011469s) [upload=474390]: Done uploading segments... Jul 18 02:19:33.736395702 (593h9m8.507037037s) [upload=474390]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration:* 30.245082596s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................XX.......X] duration: 30.25263717s FAIL (2025-07-18 02:19:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 54IOY5X3C3UH7AFBWCI22OYA, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.954507279 (593h8m8.725148614s) [noise=true , address=77.90.31.17:4772 , piece_id=VL7ZYZP75HE5FTQF5A6WRQXQKUQONXOMAIO4LWDGMSCFONDIEOGQ, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.954908767 (593h8m8.725550102s) [noise=true , address=104.36.82.2:4731 , piece_id=EO7VUDFMJ6DNAN4UDOXGCU6DBEXKPZJEN7HCZGX6NCO7MFQ5ZZHA, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.954978301 (593h8m8.725619636s) [noise=true , address=45.91.116.7:4011 , piece_id=DF2KVCWLU3L6GNAKCJCB4I23IZGCPYM7WTVWBTCWNTGILGJNA4UA, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.955164146 (593h8m8.725805481s) [noise=true , address=45.91.116.14:4190 , piece_id=ETQXF2VRB64SIGVFN6IHL7NSNB4PDXUPB3BFGDJFNTIKYXZ3SQGA, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.955328009 (593h8m8.725969344s) [noise=true , address=45.91.116.5:4298 , piece_id=YVC2CP52EATVXNKRKPZWXDETEUTWOYRKUVWDC73EZWICVENBT55A, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.955384337 (593h8m8.726025672s) [noise=true , address=45.91.116.10:4081 , piece_id=YKEG6SLWQ6FSU5AHXIGMDTY66LNFB3NN5TFW7H5A6KIEMDOD3ZPA, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.955630308 (593h8m8.726271643s) [noise=true , address=45.91.116.6:4973 , piece_id=3TROYWFQU6JNAEWYC2GJHSMDILVXG4KRGGVE7UI5RNWKMA4UM6ZA, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.955633914 (593h8m8.726275249s) [noise=true , address=45.91.116.6:4974 , piece_id=XM7YKV6AAARCVW4UTJNOPVIGPT4I2UHHUCSTQPWW27ZPGKMLBAZA, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.955906987 (593h8m8.726548332s) [noise=true , address=45.91.116.9:4261 , piece_id=XIUG3YGOAXPJ3Z67FRX4GFGAJNNZ5TX3CHGB773UWAVNF5FKZ3RQ, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.955940381 (593h8m8.726581717s) [noise=true , address=104.36.82.6:4873 , piece_id=ZIMEGNEYGSBEVKFX46LMKR2BYGLTKV2GWOHAZTFJRDB6UHCVTUKQ, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.956108874 (593h8m8.726750209s) [noise=true , address=45.91.116.7:4009 , piece_id=VGDM3NWTKLVQUV5N3ULSHWLCUOIO5QRGEE5TEXP43IEALB3ZQEMQ, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.956250344 (593h8m8.726891679s) [noise=true , address=45.91.116.13:4153 , piece_id=ZYKZVSZ4PI6KO3JHKWMEFA6ZQ742MWUT2JEGQYBM7BAXOUE3U42Q, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.956435028 (593h8m8.727076363s) [noise=true , address=104.36.82.15:4257 , piece_id=RJM2VBGSUW7ETUZ3PZ24A6FOZUNQCNV35Y3DVBMG7UWKYKKWVTQA, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.956491997 (593h8m8.727133332s) [noise=true , address=104.36.82.6:4874 , piece_id=VW425V42IDFSPZOB2TZSHN5M3LDJFAODXZWBPZLP5A5V5FGGMMPA, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:33.956686769 (593h8m8.727328104s) [noise=true , address=77.90.31.14:4661 , piece_id=S2UUXQKZ3KGDYGE3H4LONCQZWDSTXFEPLQO6SZO4CSTUN5X6T3QQ, seg_pos={0 0}, upload=474384]: Uploading piece... Jul 18 02:18:35.717157642 (593h8m10.487798967s) [upload=474384]: Next returned nil segment Jul 18 02:18:35.717173462 (593h8m10.487814797s) [upload=474384]: Waiting for error group managing segments... Jul 18 02:18:36.008431223 (593h8m10.779072558s) [noise=true , address=104.36.82.6:4873 , piece_id=ZIMEGNEYGSBEVKFX46LMKR2BYGLTKV2GWOHAZTFJRDB6UHCVTUKQ, seg_pos={0 0}, upload=474384]: Done uploading piece. err: <nil> Jul 18 02:18:36.157542677 (593h8m10.928184022s) [noise=true , address=104.36.82.2:4731 , piece_id=EO7VUDFMJ6DNAN4UDOXGCU6DBEXKPZJEN7HCZGX6NCO7MFQ5ZZHA, seg_pos={0 0}, upload=474384]: Done uploading piece. err: <nil> Jul 18 02:18:36.181102002 (593h8m10.951743337s) [noise=true , address=104.36.82.6:4874 , piece_id=VW425V42IDFSPZOB2TZSHN5M3LDJFAODXZWBPZLP5A5V5FGGMMPA, seg_pos={0 0}, upload=474384]: Done uploading piece. err: <nil> Jul 18 02:18:36.209520255 (593h8m10.98016159s) [noise=true , address=104.36.82.15:4257 , piece_id=RJM2VBGSUW7ETUZ3PZ24A6FOZUNQCNV35Y3DVBMG7UWKYKKWVTQA, seg_pos={0 0}, upload=474384]: Done uploading piece. err: <nil> Jul 18 02:19:03.002179421 (593h8m37.772820756s) [noise=true , address=45.91.116.4:4938 , piece_id=VQ6HGECLF4F63ENC2DLD4TAYZUGC54IOY5X3C3UH7AFBWCI22OYA, seg_pos={0 0}, upload=474384]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:03.002231170 (593h8m37.772872505s) [noise=true , address=45.91.116.7:4009 , piece_id=VGDM3NWTKLVQUV5N3ULSHWLCUOIO5QRGEE5TEXP43IEALB3ZQEMQ, seg_pos={0 0}, upload=474384]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:03.002379674 (593h8m37.773021009s) [noise=true , address=45.91.116.14:4190 , piece_id=ETQXF2VRB64SIGVFN6IHL7NSNB4PDXUPB3BFGDJFNTIKYXZ3SQGA, seg_pos={0 0}, upload=474384]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:03.002411154 (593h8m37.773052489s) [noise=true , address=77.90.31.14:4661 , piece_id=S2UUXQKZ3KGDYGE3H4LONCQZWDSTXFEPLQO6SZO4CSTUN5X6T3QQ, seg_pos={0 0}, upload=474384]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: rpc: dial tcp 77.90.31.14:4661: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:03.002451341 (593h8m37.773092676s) [noise=true , address=45.91.116.7:4011 , piece_id=DF2KVCWLU3L6GNAKCJCB4I23IZGCPYM7WTVWBTCWNTGILGJNA4UA, seg_pos={0 0}, upload=474384]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: write tcp 188.166.225.170:39090->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:03.002481198 (593h8m37.773122533s) [noise=true , address=45.91.116.5:4298 , piece_id=YVC2CP52EATVXNKRKPZWXDETEUTWOYRKUVWDC73EZWICVENBT55A, seg_pos={0 0}, upload=474384]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:03.002625744 (593h8m37.77326708s) [noise=true , address=45.91.116.13:4153 , piece_id=ZYKZVSZ4PI6KO3JHKWMEFA6ZQ742MWUT2JEGQYBM7BAXOUE3U42Q, seg_pos={0 0}, upload=474384]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:03.002670230 (593h8m37.773311575s) [noise=true , address=77.90.31.17:4772 , piece_id=VL7ZYZP75HE5FTQF5A6WRQXQKUQONXOMAIO4LWDGMSCFONDIEOGQ, seg_pos={0 0}, upload=474384]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:03.002720717 (593h8m37.773362062s) [noise=true , address=45.91.116.9:4261 , piece_id=XIUG3YGOAXPJ3Z67FRX4GFGAJNNZ5TX3CHGB773UWAVNF5FKZ3RQ, seg_pos={0 0}, upload=474384]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 188.166.225.170:39392->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:03.002759370 (593h8m37.773400705s) [noise=true , address=45.91.116.6:4974 , piece_id=XM7YKV6AAARCVW4UTJNOPVIGPT4I2UHHUCSTQPWW27ZPGKMLBAZA, seg_pos={0 0}, upload=474384]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:03.002792814 (593h8m37.773434149s) [noise=true , address=45.91.116.10:4081 , piece_id=YKEG6SLWQ6FSU5AHXIGMDTY66LNFB3NN5TFW7H5A6KIEMDOD3ZPA, seg_pos={0 0}, upload=474384]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: rpc: dial tcp 45.91.116.10:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:03.002840435 (593h8m37.773481771s) [noise=true , address=45.91.116.6:4973 , piece_id=3TROYWFQU6JNAEWYC2GJHSMDILVXG4KRGGVE7UI5RNWKMA4UM6ZA, seg_pos={0 0}, upload=474384]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:19:03.002897234 (593h8m37.773538569s) [seg_pos={0 0}, upload=474384]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Jul 18 02:19:03.253014872 (593h8m38.023656207s) [upload=474384]: Done uploading segments... Jul 18 02:19:03.253039900 (593h8m38.023681225s) [upload=474384]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 30.252666918s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X.X......] duration: 5.793865669s FAIL (2025-07-18 02:18:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== egmentBegin Jul 18 02:13:07.039742903 (593h2m41.810384238s) [seg_pos={0 0}, upload=474317]: Begin upload segment... Jul 18 02:13:07.039784822 (593h2m41.810426158s) [seg_pos={0 0}, upload=474317]: Done begin upload segment. Jul 18 02:13:07.039805202 (593h2m41.810446537s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039809820 (593h2m41.810451155s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039861690 (593h2m41.810503015s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040160441 (593h2m41.810801777s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040174428 (593h2m41.810815753s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040455326 (593h2m41.811096661s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040588590 (593h2m41.811229935s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040739680 (593h2m41.811381015s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040991682 (593h2m41.811633017s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041011921 (593h2m41.811653256s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041345288 (593h2m41.811986623s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041440490 (593h2m41.812081825s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041610416 (593h2m41.812251751s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041660651 (593h2m41.812301986s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041852188 (593h2m41.812493513s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.042167391 (593h2m41.812808726s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.045960459 (593h2m41.816601794s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.048048704 (593h2m41.818690039s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:08.158008049 (593h2m42.928649384s) [upload=474317]: Next returned nil segment Jul 18 02:13:08.158026534 (593h2m42.928667869s) [upload=474317]: Waiting for error group managing segments... Jul 18 02:13:08.437415627 (593h2m43.208056962s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.472093335 (593h2m43.24273467s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.510966702 (593h2m43.281608037s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.726447759 (593h2m43.497089094s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.811981590 (593h2m43.582622925s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.813294192 (593h2m43.583935527s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.828011711 (593h2m43.598653046s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.902572307 (593h2m43.673213642s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.922220870 (593h2m43.692862205s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.933219644 (593h2m43.703860979s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.011117926 (593h2m43.781759251s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.124526199 (593h2m43.895167534s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:14:05.859098849 (593h3m40.629740184s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: write tcp 188.166.225.170:36446->77.90.31.13:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859184554 (593h3m40.629825899s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1XuRKaSZC9qb9M17ctkcNkf5dSkBYpDBHgxMS56wThuw8s2R6, address:77.90.31.12:4618): protocol: write tcp 188.166.225.170:43454->77.90.31.12:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859310625 (593h3m40.62995196s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12G8MCviRyA85FwyuN3sXF9pwWQnP6mki6Ux68Eefx42BeWUcNR, address:77.90.31.16:4741): protocol: write tcp 188.166.225.170:33438->77.90.31.16:4741: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859403062 (593h3m40.630044397s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1hDDiXau6vNvsQv7oXa1rDLYMnfcLjtG4bDFVaPXiP1xwV94CD, address:77.90.31.18:4824): protocol: write tcp 188.166.225.170:52850->77.90.31.18:4824: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859472635 (593h3m40.63011398s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1rZYWQ1a2sTKnFpF28iQVrem2s8nSpXkeJzZDaS7jvTZNANdfp, address:77.90.31.21:4903): protocol: rpc: dial tcp 77.90.31.21:4903: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859566654 (593h3m40.630207989s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12FtB4bKTHntdUQxF8G5o9nT73oS2xB7fgD44BkRfKDNFyZnKCk, address:77.90.31.17:4782): protocol: write tcp 188.166.225.170:58782->77.90.31.17:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859586913 (593h3m40.630228258s) [seg_pos={0 0}, upload=474317]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 18 02:14:06.131579755 (593h3m40.9022211s) [upload=474317]: Done uploading segments... Jul 18 02:14:06.131626855 (593h3m40.90226819s) [upload=474317]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 5.793890927s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................XX.......] duration: 6.029443809s FAIL (2025-07-18 02:18:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== eded; context deadline exceeded Jul 18 02:12:25.988576605 (593h2m0.75921794s) [noise=true , address=77.90.31.11:4555 , piece_id=YNTV5NMMNXLYDXQUJVVFJ2EWR6XRSS5DF6MXEH5HPKQHYVJALNGA, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:12:25.988589700 (593h2m0.759231035s) [seg_pos={0 0}, upload=474301]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 18 02:12:26.244687258 (593h2m1.015328594s) [upload=474301]: Done uploading segments... Jul 18 02:12:26.244715342 (593h2m1.015356677s) [upload=474301]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Jul 18 02:12:38.540681010 (593h2m13.311322345s) [upload=474307]: Starting upload Jul 18 02:12:38.540766243 (593h2m13.311407578s) [upload=474307]: Done starting upload Jul 18 02:12:38.540850935 (593h2m13.31149227s) [upload=474307]: Uploading segments... Jul 18 02:12:38.541053403 (593h2m13.311694738s) [upload=474307]: Got next segment. Inline: false Jul 18 02:12:38.541085735 (593h2m13.31172707s) [upload=474307]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:12:38.541090765 (593h2m13.3117321s) [upload=474307]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:12:38.797027456 (593h2m13.567668791s) [seg_pos={0 0}, upload=474307]: Begin upload segment... Jul 18 02:12:38.797070568 (593h2m13.567711903s) [seg_pos={0 0}, upload=474307]: Done begin upload segment. Jul 18 02:12:38.797093492 (593h2m13.567734827s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797133509 (593h2m13.567774844s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797467238 (593h2m13.568108573s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797648023 (593h2m13.568289358s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797736373 (593h2m13.568377708s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798033230 (593h2m13.568674565s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798037439 (593h2m13.568678824s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798396786 (593h2m13.569038121s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798412376 (593h2m13.569053711s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798720835 (593h2m13.569362181s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798753238 (593h2m13.569394573s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799014878 (593h2m13.569656213s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799090012 (593h2m13.569731347s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799250329 (593h2m13.569891664s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.802416076 (593h2m13.573057411s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.803822136 (593h2m13.574463471s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:40.533085299 (593h2m15.303726635s) [upload=474307]: Next returned nil segment Jul 18 02:12:40.533102452 (593h2m15.303743787s) [upload=474307]: Waiting for error group managing segments... Jul 18 02:12:40.843586618 (593h2m15.614227953s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.844545052 (593h2m15.615186387s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.876330888 (593h2m15.646972223s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.883752999 (593h2m15.654394334s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.888909033 (593h2m15.659550368s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.895048040 (593h2m15.665689375s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.922290762 (593h2m15.692932097s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.959737109 (593h2m15.730378444s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.961648054 (593h2m15.732289429s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.981128606 (593h2m15.751769941s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.996899451 (593h2m15.767540826s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:41.980876083 (593h2m16.751517418s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:13:07.844077639 (593h2m42.614718974s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844130650 (593h2m42.614771985s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:55552->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844212888 (593h2m42.614854223s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 188.166.225.170:46276->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844231543 (593h2m42.614872878s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 188.166.225.170:42550->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844249387 (593h2m42.614890722s) [seg_pos={0 0}, upload=474307]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:13:08.098784357 (593h2m42.869425692s) [upload=474307]: Done uploading segments... Jul 18 02:13:08.098808874 (593h2m42.869450199s) [upload=474307]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.029483065s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................XX......] duration: 5.975460282s FAIL (2025-07-18 02:17:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== eded; context deadline exceeded Jul 18 02:12:25.988576605 (593h2m0.75921794s) [noise=true , address=77.90.31.11:4555 , piece_id=YNTV5NMMNXLYDXQUJVVFJ2EWR6XRSS5DF6MXEH5HPKQHYVJALNGA, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:12:25.988589700 (593h2m0.759231035s) [seg_pos={0 0}, upload=474301]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 18 02:12:26.244687258 (593h2m1.015328594s) [upload=474301]: Done uploading segments... Jul 18 02:12:26.244715342 (593h2m1.015356677s) [upload=474301]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Jul 18 02:12:38.540681010 (593h2m13.311322345s) [upload=474307]: Starting upload Jul 18 02:12:38.540766243 (593h2m13.311407578s) [upload=474307]: Done starting upload Jul 18 02:12:38.540850935 (593h2m13.31149227s) [upload=474307]: Uploading segments... Jul 18 02:12:38.541053403 (593h2m13.311694738s) [upload=474307]: Got next segment. Inline: false Jul 18 02:12:38.541085735 (593h2m13.31172707s) [upload=474307]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:12:38.541090765 (593h2m13.3117321s) [upload=474307]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:12:38.797027456 (593h2m13.567668791s) [seg_pos={0 0}, upload=474307]: Begin upload segment... Jul 18 02:12:38.797070568 (593h2m13.567711903s) [seg_pos={0 0}, upload=474307]: Done begin upload segment. Jul 18 02:12:38.797093492 (593h2m13.567734827s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797133509 (593h2m13.567774844s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797467238 (593h2m13.568108573s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797648023 (593h2m13.568289358s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797736373 (593h2m13.568377708s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798033230 (593h2m13.568674565s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798037439 (593h2m13.568678824s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798396786 (593h2m13.569038121s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798412376 (593h2m13.569053711s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798720835 (593h2m13.569362181s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798753238 (593h2m13.569394573s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799014878 (593h2m13.569656213s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799090012 (593h2m13.569731347s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799250329 (593h2m13.569891664s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.802416076 (593h2m13.573057411s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.803822136 (593h2m13.574463471s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:40.533085299 (593h2m15.303726635s) [upload=474307]: Next returned nil segment Jul 18 02:12:40.533102452 (593h2m15.303743787s) [upload=474307]: Waiting for error group managing segments... Jul 18 02:12:40.843586618 (593h2m15.614227953s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.844545052 (593h2m15.615186387s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.876330888 (593h2m15.646972223s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.883752999 (593h2m15.654394334s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.888909033 (593h2m15.659550368s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.895048040 (593h2m15.665689375s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.922290762 (593h2m15.692932097s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.959737109 (593h2m15.730378444s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.961648054 (593h2m15.732289429s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.981128606 (593h2m15.751769941s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.996899451 (593h2m15.767540826s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:41.980876083 (593h2m16.751517418s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:13:07.844077639 (593h2m42.614718974s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844130650 (593h2m42.614771985s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:55552->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844212888 (593h2m42.614854223s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 188.166.225.170:46276->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844231543 (593h2m42.614872878s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 188.166.225.170:42550->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844249387 (593h2m42.614890722s) [seg_pos={0 0}, upload=474307]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:13:08.098784357 (593h2m42.869425692s) [upload=474307]: Done uploading segments... Jul 18 02:13:08.098808874 (593h2m42.869450199s) [upload=474307]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.975484318s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X.X.....] duration: 5.943845444s FAIL (2025-07-18 02:16:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== egmentBegin Jul 18 02:13:07.039742903 (593h2m41.810384238s) [seg_pos={0 0}, upload=474317]: Begin upload segment... Jul 18 02:13:07.039784822 (593h2m41.810426158s) [seg_pos={0 0}, upload=474317]: Done begin upload segment. Jul 18 02:13:07.039805202 (593h2m41.810446537s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039809820 (593h2m41.810451155s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039861690 (593h2m41.810503015s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040160441 (593h2m41.810801777s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040174428 (593h2m41.810815753s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040455326 (593h2m41.811096661s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040588590 (593h2m41.811229935s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040739680 (593h2m41.811381015s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040991682 (593h2m41.811633017s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041011921 (593h2m41.811653256s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041345288 (593h2m41.811986623s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041440490 (593h2m41.812081825s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041610416 (593h2m41.812251751s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041660651 (593h2m41.812301986s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041852188 (593h2m41.812493513s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.042167391 (593h2m41.812808726s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.045960459 (593h2m41.816601794s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.048048704 (593h2m41.818690039s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:08.158008049 (593h2m42.928649384s) [upload=474317]: Next returned nil segment Jul 18 02:13:08.158026534 (593h2m42.928667869s) [upload=474317]: Waiting for error group managing segments... Jul 18 02:13:08.437415627 (593h2m43.208056962s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.472093335 (593h2m43.24273467s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.510966702 (593h2m43.281608037s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.726447759 (593h2m43.497089094s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.811981590 (593h2m43.582622925s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.813294192 (593h2m43.583935527s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.828011711 (593h2m43.598653046s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.902572307 (593h2m43.673213642s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.922220870 (593h2m43.692862205s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.933219644 (593h2m43.703860979s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.011117926 (593h2m43.781759251s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.124526199 (593h2m43.895167534s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:14:05.859098849 (593h3m40.629740184s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: write tcp 188.166.225.170:36446->77.90.31.13:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859184554 (593h3m40.629825899s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1XuRKaSZC9qb9M17ctkcNkf5dSkBYpDBHgxMS56wThuw8s2R6, address:77.90.31.12:4618): protocol: write tcp 188.166.225.170:43454->77.90.31.12:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859310625 (593h3m40.62995196s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12G8MCviRyA85FwyuN3sXF9pwWQnP6mki6Ux68Eefx42BeWUcNR, address:77.90.31.16:4741): protocol: write tcp 188.166.225.170:33438->77.90.31.16:4741: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859403062 (593h3m40.630044397s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1hDDiXau6vNvsQv7oXa1rDLYMnfcLjtG4bDFVaPXiP1xwV94CD, address:77.90.31.18:4824): protocol: write tcp 188.166.225.170:52850->77.90.31.18:4824: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859472635 (593h3m40.63011398s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1rZYWQ1a2sTKnFpF28iQVrem2s8nSpXkeJzZDaS7jvTZNANdfp, address:77.90.31.21:4903): protocol: rpc: dial tcp 77.90.31.21:4903: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859566654 (593h3m40.630207989s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12FtB4bKTHntdUQxF8G5o9nT73oS2xB7fgD44BkRfKDNFyZnKCk, address:77.90.31.17:4782): protocol: write tcp 188.166.225.170:58782->77.90.31.17:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859586913 (593h3m40.630228258s) [seg_pos={0 0}, upload=474317]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 18 02:14:06.131579755 (593h3m40.9022211s) [upload=474317]: Done uploading segments... Jul 18 02:14:06.131626855 (593h3m40.90226819s) [upload=474317]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 5.943930758s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration: 6.014561204s FAIL (2025-07-18 02:16:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== eded; context deadline exceeded Jul 18 02:12:25.988576605 (593h2m0.75921794s) [noise=true , address=77.90.31.11:4555 , piece_id=YNTV5NMMNXLYDXQUJVVFJ2EWR6XRSS5DF6MXEH5HPKQHYVJALNGA, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:12:25.988589700 (593h2m0.759231035s) [seg_pos={0 0}, upload=474301]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 18 02:12:26.244687258 (593h2m1.015328594s) [upload=474301]: Done uploading segments... Jul 18 02:12:26.244715342 (593h2m1.015356677s) [upload=474301]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Jul 18 02:12:38.540681010 (593h2m13.311322345s) [upload=474307]: Starting upload Jul 18 02:12:38.540766243 (593h2m13.311407578s) [upload=474307]: Done starting upload Jul 18 02:12:38.540850935 (593h2m13.31149227s) [upload=474307]: Uploading segments... Jul 18 02:12:38.541053403 (593h2m13.311694738s) [upload=474307]: Got next segment. Inline: false Jul 18 02:12:38.541085735 (593h2m13.31172707s) [upload=474307]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:12:38.541090765 (593h2m13.3117321s) [upload=474307]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:12:38.797027456 (593h2m13.567668791s) [seg_pos={0 0}, upload=474307]: Begin upload segment... Jul 18 02:12:38.797070568 (593h2m13.567711903s) [seg_pos={0 0}, upload=474307]: Done begin upload segment. Jul 18 02:12:38.797093492 (593h2m13.567734827s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797133509 (593h2m13.567774844s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797467238 (593h2m13.568108573s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797648023 (593h2m13.568289358s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797736373 (593h2m13.568377708s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798033230 (593h2m13.568674565s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798037439 (593h2m13.568678824s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798396786 (593h2m13.569038121s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798412376 (593h2m13.569053711s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798720835 (593h2m13.569362181s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798753238 (593h2m13.569394573s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799014878 (593h2m13.569656213s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799090012 (593h2m13.569731347s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799250329 (593h2m13.569891664s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.802416076 (593h2m13.573057411s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.803822136 (593h2m13.574463471s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:40.533085299 (593h2m15.303726635s) [upload=474307]: Next returned nil segment Jul 18 02:12:40.533102452 (593h2m15.303743787s) [upload=474307]: Waiting for error group managing segments... Jul 18 02:12:40.843586618 (593h2m15.614227953s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.844545052 (593h2m15.615186387s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.876330888 (593h2m15.646972223s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.883752999 (593h2m15.654394334s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.888909033 (593h2m15.659550368s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.895048040 (593h2m15.665689375s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.922290762 (593h2m15.692932097s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.959737109 (593h2m15.730378444s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.961648054 (593h2m15.732289429s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.981128606 (593h2m15.751769941s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.996899451 (593h2m15.767540826s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:41.980876083 (593h2m16.751517418s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:13:07.844077639 (593h2m42.614718974s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844130650 (593h2m42.614771985s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:55552->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844212888 (593h2m42.614854223s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 188.166.225.170:46276->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844231543 (593h2m42.614872878s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 188.166.225.170:42550->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844249387 (593h2m42.614890722s) [seg_pos={0 0}, upload=474307]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:13:08.098784357 (593h2m42.869425692s) [upload=474307]: Done uploading segments... Jul 18 02:13:08.098808874 (593h2m42.869450199s) [upload=474307]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.014581562s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X.X....] duration: 5.888244857s FAIL (2025-07-18 02:16:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== egmentBegin Jul 18 02:13:07.039742903 (593h2m41.810384238s) [seg_pos={0 0}, upload=474317]: Begin upload segment... Jul 18 02:13:07.039784822 (593h2m41.810426158s) [seg_pos={0 0}, upload=474317]: Done begin upload segment. Jul 18 02:13:07.039805202 (593h2m41.810446537s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039809820 (593h2m41.810451155s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039861690 (593h2m41.810503015s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040160441 (593h2m41.810801777s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040174428 (593h2m41.810815753s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040455326 (593h2m41.811096661s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040588590 (593h2m41.811229935s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040739680 (593h2m41.811381015s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040991682 (593h2m41.811633017s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041011921 (593h2m41.811653256s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041345288 (593h2m41.811986623s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041440490 (593h2m41.812081825s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041610416 (593h2m41.812251751s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041660651 (593h2m41.812301986s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041852188 (593h2m41.812493513s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.042167391 (593h2m41.812808726s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.045960459 (593h2m41.816601794s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.048048704 (593h2m41.818690039s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:08.158008049 (593h2m42.928649384s) [upload=474317]: Next returned nil segment Jul 18 02:13:08.158026534 (593h2m42.928667869s) [upload=474317]: Waiting for error group managing segments... Jul 18 02:13:08.437415627 (593h2m43.208056962s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.472093335 (593h2m43.24273467s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.510966702 (593h2m43.281608037s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.726447759 (593h2m43.497089094s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.811981590 (593h2m43.582622925s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.813294192 (593h2m43.583935527s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.828011711 (593h2m43.598653046s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.902572307 (593h2m43.673213642s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.922220870 (593h2m43.692862205s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.933219644 (593h2m43.703860979s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.011117926 (593h2m43.781759251s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.124526199 (593h2m43.895167534s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:14:05.859098849 (593h3m40.629740184s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: write tcp 188.166.225.170:36446->77.90.31.13:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859184554 (593h3m40.629825899s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1XuRKaSZC9qb9M17ctkcNkf5dSkBYpDBHgxMS56wThuw8s2R6, address:77.90.31.12:4618): protocol: write tcp 188.166.225.170:43454->77.90.31.12:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859310625 (593h3m40.62995196s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12G8MCviRyA85FwyuN3sXF9pwWQnP6mki6Ux68Eefx42BeWUcNR, address:77.90.31.16:4741): protocol: write tcp 188.166.225.170:33438->77.90.31.16:4741: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859403062 (593h3m40.630044397s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1hDDiXau6vNvsQv7oXa1rDLYMnfcLjtG4bDFVaPXiP1xwV94CD, address:77.90.31.18:4824): protocol: write tcp 188.166.225.170:52850->77.90.31.18:4824: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859472635 (593h3m40.63011398s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1rZYWQ1a2sTKnFpF28iQVrem2s8nSpXkeJzZDaS7jvTZNANdfp, address:77.90.31.21:4903): protocol: rpc: dial tcp 77.90.31.21:4903: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859566654 (593h3m40.630207989s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12FtB4bKTHntdUQxF8G5o9nT73oS2xB7fgD44BkRfKDNFyZnKCk, address:77.90.31.17:4782): protocol: write tcp 188.166.225.170:58782->77.90.31.17:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859586913 (593h3m40.630228258s) [seg_pos={0 0}, upload=474317]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 18 02:14:06.131579755 (593h3m40.9022211s) [upload=474317]: Done uploading segments... Jul 18 02:14:06.131626855 (593h3m40.90226819s) [upload=474317]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 5.888271847s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................XX....] duration: 6.015675466s FAIL (2025-07-18 02:15:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== eded; context deadline exceeded Jul 18 02:12:25.988576605 (593h2m0.75921794s) [noise=true , address=77.90.31.11:4555 , piece_id=YNTV5NMMNXLYDXQUJVVFJ2EWR6XRSS5DF6MXEH5HPKQHYVJALNGA, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:12:25.988589700 (593h2m0.759231035s) [seg_pos={0 0}, upload=474301]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 18 02:12:26.244687258 (593h2m1.015328594s) [upload=474301]: Done uploading segments... Jul 18 02:12:26.244715342 (593h2m1.015356677s) [upload=474301]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Jul 18 02:12:38.540681010 (593h2m13.311322345s) [upload=474307]: Starting upload Jul 18 02:12:38.540766243 (593h2m13.311407578s) [upload=474307]: Done starting upload Jul 18 02:12:38.540850935 (593h2m13.31149227s) [upload=474307]: Uploading segments... Jul 18 02:12:38.541053403 (593h2m13.311694738s) [upload=474307]: Got next segment. Inline: false Jul 18 02:12:38.541085735 (593h2m13.31172707s) [upload=474307]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:12:38.541090765 (593h2m13.3117321s) [upload=474307]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:12:38.797027456 (593h2m13.567668791s) [seg_pos={0 0}, upload=474307]: Begin upload segment... Jul 18 02:12:38.797070568 (593h2m13.567711903s) [seg_pos={0 0}, upload=474307]: Done begin upload segment. Jul 18 02:12:38.797093492 (593h2m13.567734827s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797133509 (593h2m13.567774844s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797467238 (593h2m13.568108573s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797648023 (593h2m13.568289358s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797736373 (593h2m13.568377708s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798033230 (593h2m13.568674565s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798037439 (593h2m13.568678824s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798396786 (593h2m13.569038121s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798412376 (593h2m13.569053711s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798720835 (593h2m13.569362181s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798753238 (593h2m13.569394573s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799014878 (593h2m13.569656213s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799090012 (593h2m13.569731347s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799250329 (593h2m13.569891664s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.802416076 (593h2m13.573057411s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.803822136 (593h2m13.574463471s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:40.533085299 (593h2m15.303726635s) [upload=474307]: Next returned nil segment Jul 18 02:12:40.533102452 (593h2m15.303743787s) [upload=474307]: Waiting for error group managing segments... Jul 18 02:12:40.843586618 (593h2m15.614227953s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.844545052 (593h2m15.615186387s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.876330888 (593h2m15.646972223s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.883752999 (593h2m15.654394334s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.888909033 (593h2m15.659550368s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.895048040 (593h2m15.665689375s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.922290762 (593h2m15.692932097s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.959737109 (593h2m15.730378444s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.961648054 (593h2m15.732289429s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.981128606 (593h2m15.751769941s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.996899451 (593h2m15.767540826s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:41.980876083 (593h2m16.751517418s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:13:07.844077639 (593h2m42.614718974s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844130650 (593h2m42.614771985s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:55552->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844212888 (593h2m42.614854223s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 188.166.225.170:46276->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844231543 (593h2m42.614872878s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 188.166.225.170:42550->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844249387 (593h2m42.614890722s) [seg_pos={0 0}, upload=474307]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:13:08.098784357 (593h2m42.869425692s) [upload=474307]: Done uploading segments... Jul 18 02:13:08.098808874 (593h2m42.869450199s) [upload=474307]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.015699683s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X.X...] duration: 6.107336962s FAIL (2025-07-18 02:15:39 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== egmentBegin Jul 18 02:13:07.039742903 (593h2m41.810384238s) [seg_pos={0 0}, upload=474317]: Begin upload segment... Jul 18 02:13:07.039784822 (593h2m41.810426158s) [seg_pos={0 0}, upload=474317]: Done begin upload segment. Jul 18 02:13:07.039805202 (593h2m41.810446537s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039809820 (593h2m41.810451155s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039861690 (593h2m41.810503015s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040160441 (593h2m41.810801777s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040174428 (593h2m41.810815753s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040455326 (593h2m41.811096661s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040588590 (593h2m41.811229935s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040739680 (593h2m41.811381015s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040991682 (593h2m41.811633017s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041011921 (593h2m41.811653256s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041345288 (593h2m41.811986623s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041440490 (593h2m41.812081825s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041610416 (593h2m41.812251751s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041660651 (593h2m41.812301986s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041852188 (593h2m41.812493513s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.042167391 (593h2m41.812808726s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.045960459 (593h2m41.816601794s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.048048704 (593h2m41.818690039s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:08.158008049 (593h2m42.928649384s) [upload=474317]: Next returned nil segment Jul 18 02:13:08.158026534 (593h2m42.928667869s) [upload=474317]: Waiting for error group managing segments... Jul 18 02:13:08.437415627 (593h2m43.208056962s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.472093335 (593h2m43.24273467s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.510966702 (593h2m43.281608037s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.726447759 (593h2m43.497089094s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.811981590 (593h2m43.582622925s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.813294192 (593h2m43.583935527s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.828011711 (593h2m43.598653046s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.902572307 (593h2m43.673213642s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.922220870 (593h2m43.692862205s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.933219644 (593h2m43.703860979s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.011117926 (593h2m43.781759251s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.124526199 (593h2m43.895167534s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:14:05.859098849 (593h3m40.629740184s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: write tcp 188.166.225.170:36446->77.90.31.13:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859184554 (593h3m40.629825899s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1XuRKaSZC9qb9M17ctkcNkf5dSkBYpDBHgxMS56wThuw8s2R6, address:77.90.31.12:4618): protocol: write tcp 188.166.225.170:43454->77.90.31.12:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859310625 (593h3m40.62995196s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12G8MCviRyA85FwyuN3sXF9pwWQnP6mki6Ux68Eefx42BeWUcNR, address:77.90.31.16:4741): protocol: write tcp 188.166.225.170:33438->77.90.31.16:4741: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859403062 (593h3m40.630044397s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1hDDiXau6vNvsQv7oXa1rDLYMnfcLjtG4bDFVaPXiP1xwV94CD, address:77.90.31.18:4824): protocol: write tcp 188.166.225.170:52850->77.90.31.18:4824: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859472635 (593h3m40.63011398s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1rZYWQ1a2sTKnFpF28iQVrem2s8nSpXkeJzZDaS7jvTZNANdfp, address:77.90.31.21:4903): protocol: rpc: dial tcp 77.90.31.21:4903: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859566654 (593h3m40.630207989s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12FtB4bKTHntdUQxF8G5o9nT73oS2xB7fgD44BkRfKDNFyZnKCk, address:77.90.31.17:4782): protocol: write tcp 188.166.225.170:58782->77.90.31.17:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859586913 (593h3m40.630228258s) [seg_pos={0 0}, upload=474317]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 18 02:14:06.131579755 (593h3m40.9022211s) [upload=474317]: Done uploading segments... Jul 18 02:14:06.131626855 (593h3m40.90226819s) [upload=474317]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 6.107364685s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................XX...] duration: 5.964444578s FAIL (2025-07-18 02:15:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== eded; context deadline exceeded Jul 18 02:12:25.988576605 (593h2m0.75921794s) [noise=true , address=77.90.31.11:4555 , piece_id=YNTV5NMMNXLYDXQUJVVFJ2EWR6XRSS5DF6MXEH5HPKQHYVJALNGA, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:12:25.988589700 (593h2m0.759231035s) [seg_pos={0 0}, upload=474301]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 18 02:12:26.244687258 (593h2m1.015328594s) [upload=474301]: Done uploading segments... Jul 18 02:12:26.244715342 (593h2m1.015356677s) [upload=474301]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Jul 18 02:12:38.540681010 (593h2m13.311322345s) [upload=474307]: Starting upload Jul 18 02:12:38.540766243 (593h2m13.311407578s) [upload=474307]: Done starting upload Jul 18 02:12:38.540850935 (593h2m13.31149227s) [upload=474307]: Uploading segments... Jul 18 02:12:38.541053403 (593h2m13.311694738s) [upload=474307]: Got next segment. Inline: false Jul 18 02:12:38.541085735 (593h2m13.31172707s) [upload=474307]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:12:38.541090765 (593h2m13.3117321s) [upload=474307]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:12:38.797027456 (593h2m13.567668791s) [seg_pos={0 0}, upload=474307]: Begin upload segment... Jul 18 02:12:38.797070568 (593h2m13.567711903s) [seg_pos={0 0}, upload=474307]: Done begin upload segment. Jul 18 02:12:38.797093492 (593h2m13.567734827s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797133509 (593h2m13.567774844s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797467238 (593h2m13.568108573s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797648023 (593h2m13.568289358s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797736373 (593h2m13.568377708s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798033230 (593h2m13.568674565s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798037439 (593h2m13.568678824s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798396786 (593h2m13.569038121s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798412376 (593h2m13.569053711s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798720835 (593h2m13.569362181s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798753238 (593h2m13.569394573s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799014878 (593h2m13.569656213s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799090012 (593h2m13.569731347s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799250329 (593h2m13.569891664s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.802416076 (593h2m13.573057411s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.803822136 (593h2m13.574463471s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:40.533085299 (593h2m15.303726635s) [upload=474307]: Next returned nil segment Jul 18 02:12:40.533102452 (593h2m15.303743787s) [upload=474307]: Waiting for error group managing segments... Jul 18 02:12:40.843586618 (593h2m15.614227953s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.844545052 (593h2m15.615186387s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.876330888 (593h2m15.646972223s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.883752999 (593h2m15.654394334s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.888909033 (593h2m15.659550368s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.895048040 (593h2m15.665689375s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.922290762 (593h2m15.692932097s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.959737109 (593h2m15.730378444s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.961648054 (593h2m15.732289429s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.981128606 (593h2m15.751769941s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.996899451 (593h2m15.767540826s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:41.980876083 (593h2m16.751517418s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:13:07.844077639 (593h2m42.614718974s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844130650 (593h2m42.614771985s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:55552->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844212888 (593h2m42.614854223s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 188.166.225.170:46276->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844231543 (593h2m42.614872878s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 188.166.225.170:42550->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844249387 (593h2m42.614890722s) [seg_pos={0 0}, upload=474307]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:13:08.098784357 (593h2m42.869425692s) [upload=474307]: Done uploading segments... Jul 18 02:13:08.098808874 (593h2m42.869450199s) [upload=474307]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.964465848s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................X.X..] duration: 6.080164983s FAIL (2025-07-18 02:14:45 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== egmentBegin Jul 18 02:13:07.039742903 (593h2m41.810384238s) [seg_pos={0 0}, upload=474317]: Begin upload segment... Jul 18 02:13:07.039784822 (593h2m41.810426158s) [seg_pos={0 0}, upload=474317]: Done begin upload segment. Jul 18 02:13:07.039805202 (593h2m41.810446537s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039809820 (593h2m41.810451155s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039861690 (593h2m41.810503015s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040160441 (593h2m41.810801777s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040174428 (593h2m41.810815753s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040455326 (593h2m41.811096661s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040588590 (593h2m41.811229935s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040739680 (593h2m41.811381015s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040991682 (593h2m41.811633017s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041011921 (593h2m41.811653256s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041345288 (593h2m41.811986623s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041440490 (593h2m41.812081825s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041610416 (593h2m41.812251751s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041660651 (593h2m41.812301986s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041852188 (593h2m41.812493513s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.042167391 (593h2m41.812808726s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.045960459 (593h2m41.816601794s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.048048704 (593h2m41.818690039s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:08.158008049 (593h2m42.928649384s) [upload=474317]: Next returned nil segment Jul 18 02:13:08.158026534 (593h2m42.928667869s) [upload=474317]: Waiting for error group managing segments... Jul 18 02:13:08.437415627 (593h2m43.208056962s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.472093335 (593h2m43.24273467s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.510966702 (593h2m43.281608037s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.726447759 (593h2m43.497089094s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.811981590 (593h2m43.582622925s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.813294192 (593h2m43.583935527s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.828011711 (593h2m43.598653046s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.902572307 (593h2m43.673213642s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.922220870 (593h2m43.692862205s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.933219644 (593h2m43.703860979s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.011117926 (593h2m43.781759251s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.124526199 (593h2m43.895167534s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:14:05.859098849 (593h3m40.629740184s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: write tcp 188.166.225.170:36446->77.90.31.13:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859184554 (593h3m40.629825899s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1XuRKaSZC9qb9M17ctkcNkf5dSkBYpDBHgxMS56wThuw8s2R6, address:77.90.31.12:4618): protocol: write tcp 188.166.225.170:43454->77.90.31.12:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859310625 (593h3m40.62995196s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12G8MCviRyA85FwyuN3sXF9pwWQnP6mki6Ux68Eefx42BeWUcNR, address:77.90.31.16:4741): protocol: write tcp 188.166.225.170:33438->77.90.31.16:4741: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859403062 (593h3m40.630044397s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1hDDiXau6vNvsQv7oXa1rDLYMnfcLjtG4bDFVaPXiP1xwV94CD, address:77.90.31.18:4824): protocol: write tcp 188.166.225.170:52850->77.90.31.18:4824: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859472635 (593h3m40.63011398s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1rZYWQ1a2sTKnFpF28iQVrem2s8nSpXkeJzZDaS7jvTZNANdfp, address:77.90.31.21:4903): protocol: rpc: dial tcp 77.90.31.21:4903: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859566654 (593h3m40.630207989s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12FtB4bKTHntdUQxF8G5o9nT73oS2xB7fgD44BkRfKDNFyZnKCk, address:77.90.31.17:4782): protocol: write tcp 188.166.225.170:58782->77.90.31.17:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859586913 (593h3m40.630228258s) [seg_pos={0 0}, upload=474317]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 18 02:14:06.131579755 (593h3m40.9022211s) [upload=474317]: Done uploading segments... Jul 18 02:14:06.131626855 (593h3m40.90226819s) [upload=474317]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 6.080192286s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 5.921145963s FAIL (2025-07-18 02:14:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== eded; context deadline exceeded Jul 18 02:12:25.988576605 (593h2m0.75921794s) [noise=true , address=77.90.31.11:4555 , piece_id=YNTV5NMMNXLYDXQUJVVFJ2EWR6XRSS5DF6MXEH5HPKQHYVJALNGA, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:12:25.988589700 (593h2m0.759231035s) [seg_pos={0 0}, upload=474301]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 18 02:12:26.244687258 (593h2m1.015328594s) [upload=474301]: Done uploading segments... Jul 18 02:12:26.244715342 (593h2m1.015356677s) [upload=474301]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Jul 18 02:12:38.540681010 (593h2m13.311322345s) [upload=474307]: Starting upload Jul 18 02:12:38.540766243 (593h2m13.311407578s) [upload=474307]: Done starting upload Jul 18 02:12:38.540850935 (593h2m13.31149227s) [upload=474307]: Uploading segments... Jul 18 02:12:38.541053403 (593h2m13.311694738s) [upload=474307]: Got next segment. Inline: false Jul 18 02:12:38.541085735 (593h2m13.31172707s) [upload=474307]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:12:38.541090765 (593h2m13.3117321s) [upload=474307]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:12:38.797027456 (593h2m13.567668791s) [seg_pos={0 0}, upload=474307]: Begin upload segment... Jul 18 02:12:38.797070568 (593h2m13.567711903s) [seg_pos={0 0}, upload=474307]: Done begin upload segment. Jul 18 02:12:38.797093492 (593h2m13.567734827s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797133509 (593h2m13.567774844s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797467238 (593h2m13.568108573s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797648023 (593h2m13.568289358s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797736373 (593h2m13.568377708s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798033230 (593h2m13.568674565s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798037439 (593h2m13.568678824s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798396786 (593h2m13.569038121s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798412376 (593h2m13.569053711s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798720835 (593h2m13.569362181s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798753238 (593h2m13.569394573s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799014878 (593h2m13.569656213s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799090012 (593h2m13.569731347s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799250329 (593h2m13.569891664s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.802416076 (593h2m13.573057411s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.803822136 (593h2m13.574463471s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:40.533085299 (593h2m15.303726635s) [upload=474307]: Next returned nil segment Jul 18 02:12:40.533102452 (593h2m15.303743787s) [upload=474307]: Waiting for error group managing segments... Jul 18 02:12:40.843586618 (593h2m15.614227953s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.844545052 (593h2m15.615186387s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.876330888 (593h2m15.646972223s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.883752999 (593h2m15.654394334s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.888909033 (593h2m15.659550368s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.895048040 (593h2m15.665689375s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.922290762 (593h2m15.692932097s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.959737109 (593h2m15.730378444s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.961648054 (593h2m15.732289429s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.981128606 (593h2m15.751769941s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.996899451 (593h2m15.767540826s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:41.980876083 (593h2m16.751517418s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:13:07.844077639 (593h2m42.614718974s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844130650 (593h2m42.614771985s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:55552->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844212888 (593h2m42.614854223s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 188.166.225.170:46276->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844231543 (593h2m42.614872878s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 188.166.225.170:42550->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844249387 (593h2m42.614890722s) [seg_pos={0 0}, upload=474307]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:13:08.098784357 (593h2m42.869425692s) [upload=474307]: Done uploading segments... Jul 18 02:13:08.098808874 (593h2m42.869450199s) [upload=474307]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.921166352s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................................X.X.] duration: 5.97872605s FAIL (2025-07-18 02:14:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== egmentBegin Jul 18 02:13:07.039742903 (593h2m41.810384238s) [seg_pos={0 0}, upload=474317]: Begin upload segment... Jul 18 02:13:07.039784822 (593h2m41.810426158s) [seg_pos={0 0}, upload=474317]: Done begin upload segment. Jul 18 02:13:07.039805202 (593h2m41.810446537s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039809820 (593h2m41.810451155s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039861690 (593h2m41.810503015s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040160441 (593h2m41.810801777s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040174428 (593h2m41.810815753s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040455326 (593h2m41.811096661s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040588590 (593h2m41.811229935s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040739680 (593h2m41.811381015s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040991682 (593h2m41.811633017s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041011921 (593h2m41.811653256s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041345288 (593h2m41.811986623s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041440490 (593h2m41.812081825s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041610416 (593h2m41.812251751s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041660651 (593h2m41.812301986s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041852188 (593h2m41.812493513s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.042167391 (593h2m41.812808726s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.045960459 (593h2m41.816601794s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.048048704 (593h2m41.818690039s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:08.158008049 (593h2m42.928649384s) [upload=474317]: Next returned nil segment Jul 18 02:13:08.158026534 (593h2m42.928667869s) [upload=474317]: Waiting for error group managing segments... Jul 18 02:13:08.437415627 (593h2m43.208056962s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.472093335 (593h2m43.24273467s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.510966702 (593h2m43.281608037s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.726447759 (593h2m43.497089094s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.811981590 (593h2m43.582622925s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.813294192 (593h2m43.583935527s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.828011711 (593h2m43.598653046s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.902572307 (593h2m43.673213642s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.922220870 (593h2m43.692862205s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.933219644 (593h2m43.703860979s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.011117926 (593h2m43.781759251s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.124526199 (593h2m43.895167534s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:14:05.859098849 (593h3m40.629740184s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: write tcp 188.166.225.170:36446->77.90.31.13:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859184554 (593h3m40.629825899s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1XuRKaSZC9qb9M17ctkcNkf5dSkBYpDBHgxMS56wThuw8s2R6, address:77.90.31.12:4618): protocol: write tcp 188.166.225.170:43454->77.90.31.12:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859310625 (593h3m40.62995196s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12G8MCviRyA85FwyuN3sXF9pwWQnP6mki6Ux68Eefx42BeWUcNR, address:77.90.31.16:4741): protocol: write tcp 188.166.225.170:33438->77.90.31.16:4741: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859403062 (593h3m40.630044397s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1hDDiXau6vNvsQv7oXa1rDLYMnfcLjtG4bDFVaPXiP1xwV94CD, address:77.90.31.18:4824): protocol: write tcp 188.166.225.170:52850->77.90.31.18:4824: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859472635 (593h3m40.63011398s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1rZYWQ1a2sTKnFpF28iQVrem2s8nSpXkeJzZDaS7jvTZNANdfp, address:77.90.31.21:4903): protocol: rpc: dial tcp 77.90.31.21:4903: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859566654 (593h3m40.630207989s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12FtB4bKTHntdUQxF8G5o9nT73oS2xB7fgD44BkRfKDNFyZnKCk, address:77.90.31.17:4782): protocol: write tcp 188.166.225.170:58782->77.90.31.17:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859586913 (593h3m40.630228258s) [seg_pos={0 0}, upload=474317]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 18 02:14:06.131579755 (593h3m40.9022211s) [upload=474317]: Done uploading segments... Jul 18 02:14:06.131626855 (593h3m40.90226819s) [upload=474317]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 5.978744507s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................X.X] duration: 1m0.275332473s FAIL (2025-07-18 02:14:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== egmentBegin Jul 18 02:13:07.039742903 (593h2m41.810384238s) [seg_pos={0 0}, upload=474317]: Begin upload segment... Jul 18 02:13:07.039784822 (593h2m41.810426158s) [seg_pos={0 0}, upload=474317]: Done begin upload segment. Jul 18 02:13:07.039805202 (593h2m41.810446537s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039809820 (593h2m41.810451155s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.039861690 (593h2m41.810503015s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040160441 (593h2m41.810801777s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040174428 (593h2m41.810815753s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040455326 (593h2m41.811096661s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040588590 (593h2m41.811229935s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040739680 (593h2m41.811381015s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.040991682 (593h2m41.811633017s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041011921 (593h2m41.811653256s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041345288 (593h2m41.811986623s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041440490 (593h2m41.812081825s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041610416 (593h2m41.812251751s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041660651 (593h2m41.812301986s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.041852188 (593h2m41.812493513s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.042167391 (593h2m41.812808726s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.045960459 (593h2m41.816601794s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:07.048048704 (593h2m41.818690039s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Uploading piece... Jul 18 02:13:08.158008049 (593h2m42.928649384s) [upload=474317]: Next returned nil segment Jul 18 02:13:08.158026534 (593h2m42.928667869s) [upload=474317]: Waiting for error group managing segments... Jul 18 02:13:08.437415627 (593h2m43.208056962s) [noise=true , address=79.127.205.235:4665 , piece_id=QOMEJKOEUQXKBKTKBMEKT7OL6JGGROD2I56U7PXX3TARNP2KYECA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.472093335 (593h2m43.24273467s) [noise=true , address=79.127.205.234:4592 , piece_id=NPMCXLPIKADRWXZIII62YDDBNZ55FE3BPIBMHC5XU5DGVW6T3YIQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.510966702 (593h2m43.281608037s) [noise=true , address=64.49.13.137:4504 , piece_id=USI5BXOQTXVN5LL5RQSIBQJB7MSIJOUZ2I5R5WMGE5D4LUBJKUXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.726447759 (593h2m43.497089094s) [noise=true , address=79.127.220.97:4379 , piece_id=K2TXFSXY55J3OB4GZAFUCIEUYSIR5JOVHSOB2N2NFKBVZIK5NLXQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.811981590 (593h2m43.582622925s) [noise=true , address=45.91.95.3:4008 , piece_id=SNCIL4IGM6HOILMASWTLXY6GYPATCX4GQ4NWVNAZ4QSVLEI7WQAQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.813294192 (593h2m43.583935527s) [noise=true , address=45.91.95.4:4030 , piece_id=ZEPSYVD4IYHD2GCCKIRPXOHKGLMUALKAJYQGCT6CSRKM6JAOLUFA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.828011711 (593h2m43.598653046s) [noise=true , address=45.91.95.2:4001 , piece_id=S2QPN4CQSCL7ZCIH4LJUBR45YZGONSXZNPSF7QIEIFOWBDWZQLGA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.902572307 (593h2m43.673213642s) [noise=true , address=45.91.95.3:4010 , piece_id=5RCFDGAJMUM5BM7AT46M7NESBJNPJPVHO2SICO7FQIC6CYJQFOZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.922220870 (593h2m43.692862205s) [noise=true , address=79.127.201.214:4312 , piece_id=YOHPTSNHEEG44RFSDXTQRPMV5SHALYVU656MAQQWMRQP67SGS7YQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:08.933219644 (593h2m43.703860979s) [noise=true , address=79.127.201.215:4352 , piece_id=ZLAHZ6PVLFS3BHUVGRC4J77KH3CEACKFAQUFTVQY5N7RAANQFNWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.011117926 (593h2m43.781759251s) [noise=true , address=45.91.116.12:4237 , piece_id=JXYCP5B6RRP5HYNMOWI6LVEAQ3N6ZHO3ATMM6PNCG7L3XLLQUXJQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:13:09.124526199 (593h2m43.895167534s) [noise=true , address=104.36.82.49:4992 , piece_id=PRJHXCSEV6VZRVBQG7REBKQY64WMKQ6PY67UVP3I3SAKXNQDZYCQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: <nil> Jul 18 02:14:05.859098849 (593h3m40.629740184s) [noise=true , address=77.90.31.13:4645 , piece_id=F3XWNTBWHPD2ZCMT6NACHFEMXNVI5RXV465P74D6TFCFK3SYQS6A, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1b2aaYQ4o7X5yLcXG3moJ3d4oxFoNSCwsqacwg9mshJ4MPoGdg, address:77.90.31.13:4645): protocol: write tcp 188.166.225.170:36446->77.90.31.13:4645: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859184554 (593h3m40.629825899s) [noise=true , address=77.90.31.12:4618 , piece_id=QT5SZFWMJ4ADBH5DRWTFNTH6PBRNFDBK2EW5RY2NRLKJWZHH2YYQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1XuRKaSZC9qb9M17ctkcNkf5dSkBYpDBHgxMS56wThuw8s2R6, address:77.90.31.12:4618): protocol: write tcp 188.166.225.170:43454->77.90.31.12:4618: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859310625 (593h3m40.62995196s) [noise=true , address=77.90.31.16:4741 , piece_id=2ZDUDLSELKGXVKMLKG4PDX3OTM4VTXLT4JETKURKOURVANPHJGMA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12G8MCviRyA85FwyuN3sXF9pwWQnP6mki6Ux68Eefx42BeWUcNR, address:77.90.31.16:4741): protocol: write tcp 188.166.225.170:33438->77.90.31.16:4741: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859403062 (593h3m40.630044397s) [noise=true , address=77.90.31.18:4824 , piece_id=G6Z4QM4RHXRRDAVON6ADKZBQ7GLJAO6QKI6RZ54SHQZUSMZXCWZQ, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1hDDiXau6vNvsQv7oXa1rDLYMnfcLjtG4bDFVaPXiP1xwV94CD, address:77.90.31.18:4824): protocol: write tcp 188.166.225.170:52850->77.90.31.18:4824: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859472635 (593h3m40.63011398s) [noise=true , address=77.90.31.21:4903 , piece_id=SOHFCENF7JVX2YQ2TAGMHRELGZZ4V4ZGUBY5NAK4VUZKMW3QADTA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:1rZYWQ1a2sTKnFpF28iQVrem2s8nSpXkeJzZDaS7jvTZNANdfp, address:77.90.31.21:4903): protocol: rpc: dial tcp 77.90.31.21:4903: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859566654 (593h3m40.630207989s) [noise=true , address=77.90.31.17:4782 , piece_id=3AL4HYUITVEKS3VPHNXG5J4EYDFYYUJVRWBM72YDTBZJPSRUZAWA, seg_pos={0 0}, upload=474317]: Done uploading piece. err: ecclient: upload failed (node:12FtB4bKTHntdUQxF8G5o9nT73oS2xB7fgD44BkRfKDNFyZnKCk, address:77.90.31.17:4782): protocol: write tcp 188.166.225.170:58782->77.90.31.17:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:14:05.859586913 (593h3m40.630228258s) [seg_pos={0 0}, upload=474317]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Jul 18 02:14:06.131579755 (593h3m40.9022211s) [upload=474317]: Done uploading segments... Jul 18 02:14:06.131626855 (593h3m40.90226819s) [upload=474317]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.275357632s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 22.031846646s FAIL (2025-07-18 02:13:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== eded; context deadline exceeded Jul 18 02:12:25.988576605 (593h2m0.75921794s) [noise=true , address=77.90.31.11:4555 , piece_id=YNTV5NMMNXLYDXQUJVVFJ2EWR6XRSS5DF6MXEH5HPKQHYVJALNGA, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:12:25.988589700 (593h2m0.759231035s) [seg_pos={0 0}, upload=474301]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 18 02:12:26.244687258 (593h2m1.015328594s) [upload=474301]: Done uploading segments... Jul 18 02:12:26.244715342 (593h2m1.015356677s) [upload=474301]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Jul 18 02:12:38.540681010 (593h2m13.311322345s) [upload=474307]: Starting upload Jul 18 02:12:38.540766243 (593h2m13.311407578s) [upload=474307]: Done starting upload Jul 18 02:12:38.540850935 (593h2m13.31149227s) [upload=474307]: Uploading segments... Jul 18 02:12:38.541053403 (593h2m13.311694738s) [upload=474307]: Got next segment. Inline: false Jul 18 02:12:38.541085735 (593h2m13.31172707s) [upload=474307]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:12:38.541090765 (593h2m13.3117321s) [upload=474307]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:12:38.797027456 (593h2m13.567668791s) [seg_pos={0 0}, upload=474307]: Begin upload segment... Jul 18 02:12:38.797070568 (593h2m13.567711903s) [seg_pos={0 0}, upload=474307]: Done begin upload segment. Jul 18 02:12:38.797093492 (593h2m13.567734827s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797133509 (593h2m13.567774844s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797467238 (593h2m13.568108573s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797648023 (593h2m13.568289358s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797736373 (593h2m13.568377708s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798033230 (593h2m13.568674565s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798037439 (593h2m13.568678824s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798396786 (593h2m13.569038121s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798412376 (593h2m13.569053711s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798720835 (593h2m13.569362181s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798753238 (593h2m13.569394573s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799014878 (593h2m13.569656213s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799090012 (593h2m13.569731347s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799250329 (593h2m13.569891664s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.802416076 (593h2m13.573057411s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.803822136 (593h2m13.574463471s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:40.533085299 (593h2m15.303726635s) [upload=474307]: Next returned nil segment Jul 18 02:12:40.533102452 (593h2m15.303743787s) [upload=474307]: Waiting for error group managing segments... Jul 18 02:12:40.843586618 (593h2m15.614227953s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.844545052 (593h2m15.615186387s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.876330888 (593h2m15.646972223s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.883752999 (593h2m15.654394334s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.888909033 (593h2m15.659550368s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.895048040 (593h2m15.665689375s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.922290762 (593h2m15.692932097s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.959737109 (593h2m15.730378444s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.961648054 (593h2m15.732289429s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.981128606 (593h2m15.751769941s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.996899451 (593h2m15.767540826s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:41.980876083 (593h2m16.751517418s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:13:07.844077639 (593h2m42.614718974s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844130650 (593h2m42.614771985s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:55552->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844212888 (593h2m42.614854223s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 188.166.225.170:46276->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844231543 (593h2m42.614872878s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 188.166.225.170:42550->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844249387 (593h2m42.614890722s) [seg_pos={0 0}, upload=474307]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:13:08.098784357 (593h2m42.869425692s) [upload=474307]: Done uploading segments... Jul 18 02:13:08.098808874 (593h2m42.869450199s) [upload=474307]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 22.031885581s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.257482312s FAIL (2025-07-18 02:13:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== eded; context deadline exceeded Jul 18 02:12:25.988576605 (593h2m0.75921794s) [noise=true , address=77.90.31.11:4555 , piece_id=YNTV5NMMNXLYDXQUJVVFJ2EWR6XRSS5DF6MXEH5HPKQHYVJALNGA, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:12:25.988589700 (593h2m0.759231035s) [seg_pos={0 0}, upload=474301]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 18 02:12:26.244687258 (593h2m1.015328594s) [upload=474301]: Done uploading segments... Jul 18 02:12:26.244715342 (593h2m1.015356677s) [upload=474301]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Jul 18 02:12:38.540681010 (593h2m13.311322345s) [upload=474307]: Starting upload Jul 18 02:12:38.540766243 (593h2m13.311407578s) [upload=474307]: Done starting upload Jul 18 02:12:38.540850935 (593h2m13.31149227s) [upload=474307]: Uploading segments... Jul 18 02:12:38.541053403 (593h2m13.311694738s) [upload=474307]: Got next segment. Inline: false Jul 18 02:12:38.541085735 (593h2m13.31172707s) [upload=474307]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:12:38.541090765 (593h2m13.3117321s) [upload=474307]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:12:38.797027456 (593h2m13.567668791s) [seg_pos={0 0}, upload=474307]: Begin upload segment... Jul 18 02:12:38.797070568 (593h2m13.567711903s) [seg_pos={0 0}, upload=474307]: Done begin upload segment. Jul 18 02:12:38.797093492 (593h2m13.567734827s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797133509 (593h2m13.567774844s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797467238 (593h2m13.568108573s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797648023 (593h2m13.568289358s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.797736373 (593h2m13.568377708s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798033230 (593h2m13.568674565s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798037439 (593h2m13.568678824s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798396786 (593h2m13.569038121s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798412376 (593h2m13.569053711s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798720835 (593h2m13.569362181s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.798753238 (593h2m13.569394573s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799014878 (593h2m13.569656213s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799090012 (593h2m13.569731347s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.799250329 (593h2m13.569891664s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.802416076 (593h2m13.573057411s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:38.803822136 (593h2m13.574463471s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Uploading piece... Jul 18 02:12:40.533085299 (593h2m15.303726635s) [upload=474307]: Next returned nil segment Jul 18 02:12:40.533102452 (593h2m15.303743787s) [upload=474307]: Waiting for error group managing segments... Jul 18 02:12:40.843586618 (593h2m15.614227953s) [noise=true , address=104.36.82.7:4911 , piece_id=6UEBU3I5IMUPKYSIFVTSDZ7L65VI2GKDZ7HX24UEGCKFJVRHI2XQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.844545052 (593h2m15.615186387s) [noise=true , address=104.36.82.9:4005 , piece_id=PXVKEM47VYTUAOGG2263OZJOFPXUHAHQS2ANN6OXKZNE2UIBEPAA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.876330888 (593h2m15.646972223s) [noise=true , address=45.91.116.9:4263 , piece_id=2P7LDJP74EXMNHOEIOMQEGKSTCIAJWKYHFCDWJKRHAW6HR44DGCQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.883752999 (593h2m15.654394334s) [noise=true , address=45.91.116.13:4155 , piece_id=BVJCHEK7NZM2TSUZM6GRTMRIWOB6TSQ53HE4QWY5ZKH7OTKIVJCA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.888909033 (593h2m15.659550368s) [noise=true , address=45.91.116.11:4117 , piece_id=VED67UKZCEYAHZMXMTDUVRPX7RNRUVED3M6U5WHFVMEQ2MBP6Z2Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.895048040 (593h2m15.665689375s) [noise=true , address=45.91.116.5:4297 , piece_id=3M75KGMIASJBBL33BUQ7WPK7BNH6MSGG5LX64KMRM3NFQJ2V7JGA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.922290762 (593h2m15.692932097s) [noise=true , address=104.36.82.11:4113 , piece_id=KUW6Z3XU2YBAX2RKKDYU43AEJFFB4D2CQNBKFAXE3QZXXI4PR5CQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.959737109 (593h2m15.730378444s) [noise=true , address=45.91.116.4:4938 , piece_id=X67AYDWSK4LDPB4C2BFL3QY3C5VCUOZH4GIYO57WWSC45S4TVOWQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.961648054 (593h2m15.732289429s) [noise=true , address=104.36.82.11:4114 , piece_id=TE7X25ZRISYOXGELPDH3CCQAH6O2LPJYZV4AXVNJKVBDIXURSEQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.981128606 (593h2m15.751769941s) [noise=true , address=45.91.116.10:4081 , piece_id=OMTWG474UINTT2WBDPQ3AJI7RHFIMDENSDOD5ZCOGSFSN3TURPYQ, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:40.996899451 (593h2m15.767540826s) [noise=true , address=104.36.82.10:4079 , piece_id=MKPMA3YVQN2OWL2NYJTGD2LMNCXDP4K7AGGAVT6KA7YTN4PRURQA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:12:41.980876083 (593h2m16.751517418s) [noise=true , address=104.36.82.8:4970 , piece_id=YRLHAU6KI7ZLXAT2ZH72WGSOQHPDYEA6L5VYVLE4AEOWRIKK26VA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: <nil> Jul 18 02:13:07.844077639 (593h2m42.614718974s) [noise=true , address=77.90.31.17:4770 , piece_id=PVTYYUNPVFMWZH3JVE3REZ2LEJQGILVUKTT4R7G2AI3S73KZZFFA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: rpc: dial tcp 77.90.31.17:4770: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844130650 (593h2m42.614771985s) [noise=true , address=77.90.31.18:4807 , piece_id=K3ZK5TESZGL2J35DLRHUE4FKFHTYBOL67HN4V7FJ5TAZM7S7FY3Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: write tcp 188.166.225.170:55552->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844212888 (593h2m42.614854223s) [noise=true , address=77.90.31.13:4628 , piece_id=4OFIAFALB24F3IKIUSXSSOFZG4E7NTZ7UPWGQRHWH4HAL2PL7O6Q, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 188.166.225.170:46276->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844231543 (593h2m42.614872878s) [noise=true , address=77.90.31.21:4877 , piece_id=ISNPP6CV3IOKRSEX4SLHYI6OBULWMJMBJ254QUII2UBR7BHT4ESA, seg_pos={0 0}, upload=474307]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: write tcp 188.166.225.170:42550->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:13:07.844249387 (593h2m42.614890722s) [seg_pos={0 0}, upload=474307]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 02:13:08.098784357 (593h2m42.869425692s) [upload=474307]: Done uploading segments... Jul 18 02:13:08.098808874 (593h2m42.869450199s) [upload=474307]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.257532687s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.743505366s FAIL (2025-07-18 02:12:49 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== =79.127.201.214:4321 , piece_id=NC7U276TVHN65MJDB4MO3GETZS5X6P5SSMV7CLPUOTF75HCVVAQA, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.425125489 (593h1m6.195766824s) [noise=true , address=79.127.219.38:4035 , piece_id=7KULUPZINRQ76OBY5KZHUO5QQWGF4HWU7BBBJEARSOPYZQB44SEQ, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.425459768 (593h1m6.196101104s) [noise=true , address=45.91.116.13:4159 , piece_id=WGLQ5FDYQMVUDVAAJAD6BY73RWGQMEDOV3DGRN4TESKETLIMP6PQ, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.425469438 (593h1m6.196110773s) [noise=true , address=45.91.95.5:4023 , piece_id=F6CBUDVVIF4VDBBCLLKT3Y6MJWLEJPCP4FYGX7NR2NFUDB5SRXHQ, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.425724976 (593h1m6.196366311s) [noise=true , address=104.36.82.12:4163 , piece_id=X4OWLG66PX7LK3YEYIFXXBXBJHVTY7AN5GZYZ6EADPPB5R2QKVPQ, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.425988731 (593h1m6.196630066s) [noise=true , address=104.36.82.11:4132 , piece_id=ACM3LQPXJRXL4YBJWSWHZMK2U7SWEAREZSRAJIINNPL7ENYWOT6Q, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.426234131 (593h1m6.196875466s) [noise=true , address=77.90.31.21:4898 , piece_id=X6XLORVGONNSUTH56CH6U3VX43S3YSKDADCHLA2JW3M62WJMT63Q, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.426447589 (593h1m6.197088924s) [noise=true , address=77.90.31.17:4792 , piece_id=2H4DAPNJTXD26F2GYG5X6XNO4HMPSNJYCGRCWW7YWJ3D6IGITTAA, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.426658072 (593h1m6.197299407s) [noise=true , address=104.36.82.54:4182 , piece_id=5VJQZZBBTIJZJHDJDGT7CTS67O6L4X4F34LIZS7NIFUQYAT6KIMA, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.426863965 (593h1m6.1975053s) [noise=true , address=77.90.31.15:4723 , piece_id=SWNCWOULWR4A47G66QAERST7SJ3JJYEQFLGVLXH4PV5E3ILHKSPQ, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.427148330 (593h1m6.197789665s) [noise=true , address=45.91.95.3:4004 , piece_id=T56FAL55VUOCRM7KD6B3UGIGVVXJUHQOS3GYSR5ND5XLNLP6XF2Q, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.427353522 (593h1m6.197994857s) [noise=true , address=45.91.95.4:4003 , piece_id=E2ZVZZ72L3FDEDOOJYIVUJKPOM2SZS3FA2LCSQD4YXFKADBGGEOA, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.427543676 (593h1m6.198185011s) [noise=true , address=45.91.116.12:4245 , piece_id=UJGERIJCCBLK3AANYM2EZIFCCN5GRU4PY4ZL5IZJJKBVH67EQL6A, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.427729942 (593h1m6.198371278s) [noise=true , address=77.90.31.16:4758 , piece_id=F7644IT23WCFEMKKOAMPK7GYPGELCHZRPKE43TUVVO64MI2OR77Q, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.427965493 (593h1m6.198606828s) [noise=true , address=45.91.95.2:4013 , piece_id=F5KOKAH4I3CXM4WEM5C43VIYU7APJEQS6AEEWCPOVNAQV7MZYXZQ, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.428276989 (593h1m6.198918324s) [noise=true , address=79.127.205.246:4288 , piece_id=5FFWZJYJBD7IQMJHUSO7K6U33BWU2RSUH233E22NNMMVMN7L5SUA, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.428871646 (593h1m6.199512981s) [noise=true , address=77.90.31.18:4838 , piece_id=Z2TAFNL6LLUQGOEFFBZEY5FV3LGTO3LRTAZVB23U3WMWHW6EJRIA, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.429444452 (593h1m6.200085788s) [noise=true , address=77.90.31.19:4850 , piece_id=VPEAD6H73DXYCLMX7VGS4WZYVJ2DN26ZKTKWWSHH3KICZIVICY2A, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:32.537543488 (593h1m7.308184823s) [upload=474295]: Next returned nil segment Jul 18 02:11:32.537558598 (593h1m7.308199933s) [upload=474295]: Waiting for error group managing segments... Jul 18 02:11:32.833669734 (593h1m7.604311069s) [noise=true , address=79.127.205.246:4288 , piece_id=5FFWZJYJBD7IQMJHUSO7K6U33BWU2RSUH233E22NNMMVMN7L5SUA, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.112796735 (593h1m7.88343807s) [noise=true , address=45.91.95.2:4013 , piece_id=F5KOKAH4I3CXM4WEM5C43VIYU7APJEQS6AEEWCPOVNAQV7MZYXZQ, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.132402026 (593h1m7.903043361s) [noise=true , address=45.91.95.3:4004 , piece_id=T56FAL55VUOCRM7KD6B3UGIGVVXJUHQOS3GYSR5ND5XLNLP6XF2Q, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.143217068 (593h1m7.913858403s) [noise=true , address=45.91.95.5:4023 , piece_id=F6CBUDVVIF4VDBBCLLKT3Y6MJWLEJPCP4FYGX7NR2NFUDB5SRXHQ, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.323121337 (593h1m8.093762662s) [noise=true , address=79.127.201.214:4321 , piece_id=NC7U276TVHN65MJDB4MO3GETZS5X6P5SSMV7CLPUOTF75HCVVAQA, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.338556991 (593h1m8.109198326s) [noise=true , address=79.127.219.38:4035 , piece_id=7KULUPZINRQ76OBY5KZHUO5QQWGF4HWU7BBBJEARSOPYZQB44SEQ, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.392554954 (593h1m8.163196289s) [noise=true , address=45.91.95.4:4003 , piece_id=E2ZVZZ72L3FDEDOOJYIVUJKPOM2SZS3FA2LCSQD4YXFKADBGGEOA, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.446596852 (593h1m8.217238197s) [noise=true , address=104.36.82.12:4163 , piece_id=X4OWLG66PX7LK3YEYIFXXBXBJHVTY7AN5GZYZ6EADPPB5R2QKVPQ, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.521451300 (593h1m8.292092635s) [noise=true , address=104.36.82.54:4182 , piece_id=5VJQZZBBTIJZJHDJDGT7CTS67O6L4X4F34LIZS7NIFUQYAT6KIMA, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.718973818 (593h1m8.489615153s) [noise=true , address=104.36.82.11:4132 , piece_id=ACM3LQPXJRXL4YBJWSWHZMK2U7SWEAREZSRAJIINNPL7ENYWOT6Q, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:12:30.253105899 (593h2m5.023747234s) [noise=true , address=77.90.31.19:4850 , piece_id=VPEAD6H73DXYCLMX7VGS4WZYVJ2DN26ZKTKWWSHH3KICZIVICY2A, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:12oGsNdVAKAabvVuWFZ2mpydVYjzhJycig4cqdcE5oAS2YsHmYv, address:77.90.31.19:4850): protocol: write tcp 188.166.225.170:60870->77.90.31.19:4850: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253156075 (593h2m5.02379741s) [noise=true , address=77.90.31.21:4898 , piece_id=X6XLORVGONNSUTH56CH6U3VX43S3YSKDADCHLA2JW3M62WJMT63Q, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx, address:77.90.31.21:4898): protocol: write tcp 188.166.225.170:55496->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253290012 (593h2m5.023931357s) [noise=true , address=45.91.116.12:4245 , piece_id=UJGERIJCCBLK3AANYM2EZIFCCN5GRU4PY4ZL5IZJJKBVH67EQL6A, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:12Zs9PFmdpQVHe1UjYapzPRUpsopENKECrNUhVVBZ5nmhpGA9Ws, address:45.91.116.12:4245): protocol: rpc: dial tcp 45.91.116.12:4245: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253340287 (593h2m5.023981622s) [noise=true , address=77.90.31.16:4758 , piece_id=F7644IT23WCFEMKKOAMPK7GYPGELCHZRPKE43TUVVO64MI2OR77Q, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:1ahTbggGRi6cJa4JPBfSEgLJYymrgVuLxSc15BV3qj5gPWPFiZ, address:77.90.31.16:4758): protocol: rpc: dial tcp 77.90.31.16:4758: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253378941 (593h2m5.024020287s) [noise=true , address=45.91.116.13:4159 , piece_id=WGLQ5FDYQMVUDVAAJAD6BY73RWGQMEDOV3DGRN4TESKETLIMP6PQ, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253446160 (593h2m5.024087505s) [noise=true , address=77.90.31.15:4723 , piece_id=SWNCWOULWR4A47G66QAERST7SJ3JJYEQFLGVLXH4PV5E3ILHKSPQ, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:1EGKHmd15augxZS76bCbS9f7Kzj8CQSkX1tvSCapydjVqKJseS, address:77.90.31.15:4723): protocol: rpc: dial tcp 77.90.31.15:4723: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253488762 (593h2m5.024130097s) [noise=true , address=77.90.31.18:4838 , piece_id=Z2TAFNL6LLUQGOEFFBZEY5FV3LGTO3LRTAZVB23U3WMWHW6EJRIA, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: rpc: dial tcp 77.90.31.18:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253524039 (593h2m5.024165374s) [noise=true , address=77.90.31.17:4792 , piece_id=2H4DAPNJTXD26F2GYG5X6XNO4HMPSNJYCGRCWW7YWJ3D6IGITTAA, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:12SPFgYZnbwjEupjaghfs61qwFM678KNPwpDmRNqV1j2opNgZCN, address:77.90.31.17:4792): protocol: rpc: dial tcp 77.90.31.17:4792: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253540571 (593h2m5.024181906s) [seg_pos={0 0}, upload=474295]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 18 02:12:30.504635949 (593h2m5.275277284s) [upload=474295]: Done uploading segments... Jul 18 02:12:30.504668301 (593h2m5.275309656s) [upload=474295]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.743534992s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.253290943s FAIL (2025-07-18 02:12:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== =79.127.201.214:4321 , piece_id=NC7U276TVHN65MJDB4MO3GETZS5X6P5SSMV7CLPUOTF75HCVVAQA, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.425125489 (593h1m6.195766824s) [noise=true , address=79.127.219.38:4035 , piece_id=7KULUPZINRQ76OBY5KZHUO5QQWGF4HWU7BBBJEARSOPYZQB44SEQ, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.425459768 (593h1m6.196101104s) [noise=true , address=45.91.116.13:4159 , piece_id=WGLQ5FDYQMVUDVAAJAD6BY73RWGQMEDOV3DGRN4TESKETLIMP6PQ, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.425469438 (593h1m6.196110773s) [noise=true , address=45.91.95.5:4023 , piece_id=F6CBUDVVIF4VDBBCLLKT3Y6MJWLEJPCP4FYGX7NR2NFUDB5SRXHQ, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.425724976 (593h1m6.196366311s) [noise=true , address=104.36.82.12:4163 , piece_id=X4OWLG66PX7LK3YEYIFXXBXBJHVTY7AN5GZYZ6EADPPB5R2QKVPQ, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.425988731 (593h1m6.196630066s) [noise=true , address=104.36.82.11:4132 , piece_id=ACM3LQPXJRXL4YBJWSWHZMK2U7SWEAREZSRAJIINNPL7ENYWOT6Q, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.426234131 (593h1m6.196875466s) [noise=true , address=77.90.31.21:4898 , piece_id=X6XLORVGONNSUTH56CH6U3VX43S3YSKDADCHLA2JW3M62WJMT63Q, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.426447589 (593h1m6.197088924s) [noise=true , address=77.90.31.17:4792 , piece_id=2H4DAPNJTXD26F2GYG5X6XNO4HMPSNJYCGRCWW7YWJ3D6IGITTAA, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.426658072 (593h1m6.197299407s) [noise=true , address=104.36.82.54:4182 , piece_id=5VJQZZBBTIJZJHDJDGT7CTS67O6L4X4F34LIZS7NIFUQYAT6KIMA, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.426863965 (593h1m6.1975053s) [noise=true , address=77.90.31.15:4723 , piece_id=SWNCWOULWR4A47G66QAERST7SJ3JJYEQFLGVLXH4PV5E3ILHKSPQ, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.427148330 (593h1m6.197789665s) [noise=true , address=45.91.95.3:4004 , piece_id=T56FAL55VUOCRM7KD6B3UGIGVVXJUHQOS3GYSR5ND5XLNLP6XF2Q, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.427353522 (593h1m6.197994857s) [noise=true , address=45.91.95.4:4003 , piece_id=E2ZVZZ72L3FDEDOOJYIVUJKPOM2SZS3FA2LCSQD4YXFKADBGGEOA, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.427543676 (593h1m6.198185011s) [noise=true , address=45.91.116.12:4245 , piece_id=UJGERIJCCBLK3AANYM2EZIFCCN5GRU4PY4ZL5IZJJKBVH67EQL6A, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.427729942 (593h1m6.198371278s) [noise=true , address=77.90.31.16:4758 , piece_id=F7644IT23WCFEMKKOAMPK7GYPGELCHZRPKE43TUVVO64MI2OR77Q, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.427965493 (593h1m6.198606828s) [noise=true , address=45.91.95.2:4013 , piece_id=F5KOKAH4I3CXM4WEM5C43VIYU7APJEQS6AEEWCPOVNAQV7MZYXZQ, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.428276989 (593h1m6.198918324s) [noise=true , address=79.127.205.246:4288 , piece_id=5FFWZJYJBD7IQMJHUSO7K6U33BWU2RSUH233E22NNMMVMN7L5SUA, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.428871646 (593h1m6.199512981s) [noise=true , address=77.90.31.18:4838 , piece_id=Z2TAFNL6LLUQGOEFFBZEY5FV3LGTO3LRTAZVB23U3WMWHW6EJRIA, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:31.429444452 (593h1m6.200085788s) [noise=true , address=77.90.31.19:4850 , piece_id=VPEAD6H73DXYCLMX7VGS4WZYVJ2DN26ZKTKWWSHH3KICZIVICY2A, seg_pos={0 0}, upload=474295]: Uploading piece... Jul 18 02:11:32.537543488 (593h1m7.308184823s) [upload=474295]: Next returned nil segment Jul 18 02:11:32.537558598 (593h1m7.308199933s) [upload=474295]: Waiting for error group managing segments... Jul 18 02:11:32.833669734 (593h1m7.604311069s) [noise=true , address=79.127.205.246:4288 , piece_id=5FFWZJYJBD7IQMJHUSO7K6U33BWU2RSUH233E22NNMMVMN7L5SUA, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.112796735 (593h1m7.88343807s) [noise=true , address=45.91.95.2:4013 , piece_id=F5KOKAH4I3CXM4WEM5C43VIYU7APJEQS6AEEWCPOVNAQV7MZYXZQ, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.132402026 (593h1m7.903043361s) [noise=true , address=45.91.95.3:4004 , piece_id=T56FAL55VUOCRM7KD6B3UGIGVVXJUHQOS3GYSR5ND5XLNLP6XF2Q, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.143217068 (593h1m7.913858403s) [noise=true , address=45.91.95.5:4023 , piece_id=F6CBUDVVIF4VDBBCLLKT3Y6MJWLEJPCP4FYGX7NR2NFUDB5SRXHQ, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.323121337 (593h1m8.093762662s) [noise=true , address=79.127.201.214:4321 , piece_id=NC7U276TVHN65MJDB4MO3GETZS5X6P5SSMV7CLPUOTF75HCVVAQA, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.338556991 (593h1m8.109198326s) [noise=true , address=79.127.219.38:4035 , piece_id=7KULUPZINRQ76OBY5KZHUO5QQWGF4HWU7BBBJEARSOPYZQB44SEQ, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.392554954 (593h1m8.163196289s) [noise=true , address=45.91.95.4:4003 , piece_id=E2ZVZZ72L3FDEDOOJYIVUJKPOM2SZS3FA2LCSQD4YXFKADBGGEOA, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.446596852 (593h1m8.217238197s) [noise=true , address=104.36.82.12:4163 , piece_id=X4OWLG66PX7LK3YEYIFXXBXBJHVTY7AN5GZYZ6EADPPB5R2QKVPQ, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.521451300 (593h1m8.292092635s) [noise=true , address=104.36.82.54:4182 , piece_id=5VJQZZBBTIJZJHDJDGT7CTS67O6L4X4F34LIZS7NIFUQYAT6KIMA, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:11:33.718973818 (593h1m8.489615153s) [noise=true , address=104.36.82.11:4132 , piece_id=ACM3LQPXJRXL4YBJWSWHZMK2U7SWEAREZSRAJIINNPL7ENYWOT6Q, seg_pos={0 0}, upload=474295]: Done uploading piece. err: <nil> Jul 18 02:12:30.253105899 (593h2m5.023747234s) [noise=true , address=77.90.31.19:4850 , piece_id=VPEAD6H73DXYCLMX7VGS4WZYVJ2DN26ZKTKWWSHH3KICZIVICY2A, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:12oGsNdVAKAabvVuWFZ2mpydVYjzhJycig4cqdcE5oAS2YsHmYv, address:77.90.31.19:4850): protocol: write tcp 188.166.225.170:60870->77.90.31.19:4850: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253156075 (593h2m5.02379741s) [noise=true , address=77.90.31.21:4898 , piece_id=X6XLORVGONNSUTH56CH6U3VX43S3YSKDADCHLA2JW3M62WJMT63Q, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:12Qi796rSqngMJNBiFji2g4eYRn5MsqCqhpGPdYPN2GN8yFGTUx, address:77.90.31.21:4898): protocol: write tcp 188.166.225.170:55496->77.90.31.21:4898: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253290012 (593h2m5.023931357s) [noise=true , address=45.91.116.12:4245 , piece_id=UJGERIJCCBLK3AANYM2EZIFCCN5GRU4PY4ZL5IZJJKBVH67EQL6A, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:12Zs9PFmdpQVHe1UjYapzPRUpsopENKECrNUhVVBZ5nmhpGA9Ws, address:45.91.116.12:4245): protocol: rpc: dial tcp 45.91.116.12:4245: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253340287 (593h2m5.023981622s) [noise=true , address=77.90.31.16:4758 , piece_id=F7644IT23WCFEMKKOAMPK7GYPGELCHZRPKE43TUVVO64MI2OR77Q, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:1ahTbggGRi6cJa4JPBfSEgLJYymrgVuLxSc15BV3qj5gPWPFiZ, address:77.90.31.16:4758): protocol: rpc: dial tcp 77.90.31.16:4758: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253378941 (593h2m5.024020287s) [noise=true , address=45.91.116.13:4159 , piece_id=WGLQ5FDYQMVUDVAAJAD6BY73RWGQMEDOV3DGRN4TESKETLIMP6PQ, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253446160 (593h2m5.024087505s) [noise=true , address=77.90.31.15:4723 , piece_id=SWNCWOULWR4A47G66QAERST7SJ3JJYEQFLGVLXH4PV5E3ILHKSPQ, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:1EGKHmd15augxZS76bCbS9f7Kzj8CQSkX1tvSCapydjVqKJseS, address:77.90.31.15:4723): protocol: rpc: dial tcp 77.90.31.15:4723: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253488762 (593h2m5.024130097s) [noise=true , address=77.90.31.18:4838 , piece_id=Z2TAFNL6LLUQGOEFFBZEY5FV3LGTO3LRTAZVB23U3WMWHW6EJRIA, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:1if7vf1cNTBwAhb6owi6tuGLYqSfa3feWhsvGSevwuwykhz2RY, address:77.90.31.18:4838): protocol: rpc: dial tcp 77.90.31.18:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253524039 (593h2m5.024165374s) [noise=true , address=77.90.31.17:4792 , piece_id=2H4DAPNJTXD26F2GYG5X6XNO4HMPSNJYCGRCWW7YWJ3D6IGITTAA, seg_pos={0 0}, upload=474295]: Done uploading piece. err: ecclient: upload failed (node:12SPFgYZnbwjEupjaghfs61qwFM678KNPwpDmRNqV1j2opNgZCN, address:77.90.31.17:4792): protocol: rpc: dial tcp 77.90.31.17:4792: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:30.253540571 (593h2m5.024181906s) [seg_pos={0 0}, upload=474295]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 18 02:12:30.504635949 (593h2m5.275277284s) [upload=474295]: Done uploading segments... Jul 18 02:12:30.504668301 (593h2m5.275309656s) [upload=474295]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.253315739s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.25798846s FAIL (2025-07-18 02:12:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== 56.686000606 (593h1m31.456641941s) [upload=474301]: Done starting upload Jul 18 02:11:56.686085999 (593h1m31.456727335s) [upload=474301]: Uploading segments... Jul 18 02:11:56.686217842 (593h1m31.456859177s) [upload=474301]: Got next segment. Inline: false Jul 18 02:11:56.686257188 (593h1m31.456898533s) [upload=474301]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 02:11:56.686262818 (593h1m31.456904153s) [upload=474301]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 02:11:56.943063074 (593h1m31.713704409s) [seg_pos={0 0}, upload=474301]: Begin upload segment... Jul 18 02:11:56.943103641 (593h1m31.713744976s) [seg_pos={0 0}, upload=474301]: Done begin upload segment. Jul 18 02:11:56.943124812 (593h1m31.713766147s) [noise=true , address=104.36.82.11:4115 , piece_id=TNEBGIQQ3TYTTRDHWMGE3S4S24F34ZDH3BQZ5LIHWYCJRLMSFRTQ, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.943153116 (593h1m31.713794451s) [noise=true , address=104.36.82.15:4259 , piece_id=2YL22GCGUBHHXDQZ6WVUEPR5NILVYM32G6QCFFS6EJLLNTDJMYFQ, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.943472758 (593h1m31.714114093s) [noise=true , address=45.91.116.12:4225 , piece_id=HHNWGTG24MYGAZSJYL2RXSA5CEQNHEKYP5N4AVABKP5Y24SQPSBA, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.943484360 (593h1m31.714125695s) [noise=true , address=104.36.82.3:4767 , piece_id=AB2FM6PM2DL226BWWTC6F67XMVOBJJFSR2ZDS7NPE3FJSRGXBSKQ, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.943718518 (593h1m31.714359853s) [noise=true , address=77.90.31.21:4877 , piece_id=7YWGHKC7M3GCWKBYYWSTBDXRUYUOSQYNFB6JF2JNFIZAAF26CR6A, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.943959660 (593h1m31.714600995s) [noise=true , address=45.91.116.14:4190 , piece_id=SIAWWV6MPZJSYWYDE4JMOEK7WVRBU2MZ4EWG7UJTGMVDCCGBCH5Q, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.943967554 (593h1m31.714608889s) [noise=true , address=104.36.82.4:4803 , piece_id=DA4FQDSE5F7HOQT6MQZWPZAGFUNM45RLDJUU43VGS7KF47UKCHEQ, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.944263020 (593h1m31.714904355s) [noise=true , address=45.91.116.7:4009 , piece_id=2EX2ZBSV2LYFLXUGI7SSI6CIG272VAZ242HGP7PLB44DUMGBXNFQ, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.944294971 (593h1m31.714936306s) [noise=true , address=104.36.82.15:4258 , piece_id=HDWAYP7UHVA2VOP4XTWP3DBKM6ARVGXCHJ4MVA6LJXJ3QEDGZU3Q, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.944491417 (593h1m31.715132752s) [noise=true , address=45.91.116.11:4117 , piece_id=PNMQVCQ5PMOGGIODOFBMQAEYYZSY6JWDPAN62CXB7CBJCASBDAUQ, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.944523298 (593h1m31.715164643s) [noise=true , address=45.91.116.8:4048 , piece_id=IMG4PBD4GF45LTNNVN7YUQTWVEKYJFYCE7CCM7HCJYNAYE5SHZSA, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.944708572 (593h1m31.715349907s) [noise=true , address=104.36.82.8:4970 , piece_id=2XKDFNQ37ZZSEKE7OCJ2C7VZ7DX3XVK75BNMA5HKNE7ZKPDOQWRQ, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.944728600 (593h1m31.715369935s) [noise=true , address=77.90.31.11:4555 , piece_id=YNTV5NMMNXLYDXQUJVVFJ2EWR6XRSS5DF6MXEH5HPKQHYVJALNGA, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.944906691 (593h1m31.715548026s) [noise=true , address=45.91.116.6:4974 , piece_id=POS5G6Q2AVJREHGPGGMF5Y6DWRTRZRSI5KDVTPKA2FFHVW4ZFZKQ, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.944927762 (593h1m31.715569097s) [noise=true , address=45.91.116.9:4262 , piece_id=XYEKCW7RO2ETJVCDDDUE4CYWPWQ4JAU7DZXZPUCNBVY5BRN5XAYQ, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:56.945144006 (593h1m31.715785351s) [noise=true , address=104.36.82.9:4005 , piece_id=4UXNC53YX6CFOGBTNGIPN3TWWICY2GY7GLOL54TR6T3SZOQJYI7A, seg_pos={0 0}, upload=474301]: Uploading piece... Jul 18 02:11:58.640334705 (593h1m33.41097604s) [upload=474301]: Next returned nil segment Jul 18 02:11:58.640350876 (593h1m33.410992212s) [upload=474301]: Waiting for error group managing segments... Jul 18 02:11:58.978584305 (593h1m33.74922564s) [noise=true , address=104.36.82.4:4803 , piece_id=DA4FQDSE5F7HOQT6MQZWPZAGFUNM45RLDJUU43VGS7KF47UKCHEQ, seg_pos={0 0}, upload=474301]: Done uploading piece. err: <nil> Jul 18 02:11:58.990454828 (593h1m33.761096163s) [noise=true , address=104.36.82.8:4970 , piece_id=2XKDFNQ37ZZSEKE7OCJ2C7VZ7DX3XVK75BNMA5HKNE7ZKPDOQWRQ, seg_pos={0 0}, upload=474301]: Done uploading piece. err: <nil> Jul 18 02:11:58.995307973 (593h1m33.765949308s) [noise=true , address=104.36.82.9:4005 , piece_id=4UXNC53YX6CFOGBTNGIPN3TWWICY2GY7GLOL54TR6T3SZOQJYI7A, seg_pos={0 0}, upload=474301]: Done uploading piece. err: <nil> Jul 18 02:11:59.004629277 (593h1m33.775270612s) [noise=true , address=104.36.82.15:4259 , piece_id=2YL22GCGUBHHXDQZ6WVUEPR5NILVYM32G6QCFFS6EJLLNTDJMYFQ, seg_pos={0 0}, upload=474301]: Done uploading piece. err: <nil> Jul 18 02:11:59.068068736 (593h1m33.838710081s) [noise=true , address=104.36.82.15:4258 , piece_id=HDWAYP7UHVA2VOP4XTWP3DBKM6ARVGXCHJ4MVA6LJXJ3QEDGZU3Q, seg_pos={0 0}, upload=474301]: Done uploading piece. err: <nil> Jul 18 02:11:59.140138005 (593h1m33.91077934s) [noise=true , address=104.36.82.3:4767 , piece_id=AB2FM6PM2DL226BWWTC6F67XMVOBJJFSR2ZDS7NPE3FJSRGXBSKQ, seg_pos={0 0}, upload=474301]: Done uploading piece. err: <nil> Jul 18 02:11:59.236167207 (593h1m34.006808542s) [noise=true , address=104.36.82.11:4115 , piece_id=TNEBGIQQ3TYTTRDHWMGE3S4S24F34ZDH3BQZ5LIHWYCJRLMSFRTQ, seg_pos={0 0}, upload=474301]: Done uploading piece. err: <nil> Jul 18 02:12:25.988259117 (593h2m0.758900452s) [noise=true , address=77.90.31.21:4877 , piece_id=7YWGHKC7M3GCWKBYYWSTBDXRUYUOSQYNFB6JF2JNFIZAAF26CR6A, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:12:25.988392783 (593h2m0.759034108s) [noise=true , address=45.91.116.8:4048 , piece_id=IMG4PBD4GF45LTNNVN7YUQTWVEKYJFYCE7CCM7HCJYNAYE5SHZSA, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: rpc: dial tcp 45.91.116.8:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:25.988472165 (593h2m0.7591135s) [noise=true , address=45.91.116.7:4009 , piece_id=2EX2ZBSV2LYFLXUGI7SSI6CIG272VAZ242HGP7PLB44DUMGBXNFQ, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:55970->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:25.988491042 (593h2m0.759132377s) [noise=true , address=45.91.116.12:4225 , piece_id=HHNWGTG24MYGAZSJYL2RXSA5CEQNHEKYP5N4AVABKP5Y24SQPSBA, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:25.988506501 (593h2m0.759147836s) [noise=true , address=45.91.116.14:4190 , piece_id=SIAWWV6MPZJSYWYDE4JMOEK7WVRBU2MZ4EWG7UJTGMVDCCGBCH5Q, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:38202->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:25.988538582 (593h2m0.759179917s) [noise=true , address=45.91.116.11:4117 , piece_id=PNMQVCQ5PMOGGIODOFBMQAEYYZSY6JWDPAN62CXB7CBJCASBDAUQ, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:25.988554092 (593h2m0.759195427s) [noise=true , address=45.91.116.6:4974 , piece_id=POS5G6Q2AVJREHGPGGMF5Y6DWRTRZRSI5KDVTPKA2FFHVW4ZFZKQ, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 02:12:25.988565473 (593h2m0.759206808s) [noise=true , address=45.91.116.9:4262 , piece_id=XYEKCW7RO2ETJVCDDDUE4CYWPWQ4JAU7DZXZPUCNBVY5BRN5XAYQ, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:12:25.988576605 (593h2m0.75921794s) [noise=true , address=77.90.31.11:4555 , piece_id=YNTV5NMMNXLYDXQUJVVFJ2EWR6XRSS5DF6MXEH5HPKQHYVJALNGA, seg_pos={0 0}, upload=474301]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: context deadline exceeded; context deadline exceeded Jul 18 02:12:25.988589700 (593h2m0.759231035s) [seg_pos={0 0}, upload=474301]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Jul 18 02:12:26.244687258 (593h2m1.015328594s) [upload=474301]: Done uploading segments... Jul 18 02:12:26.244715342 (593h2m1.015356677s) [upload=474301]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.258012285s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.662938736s FAIL (2025-07-18 01:54:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== ress=77.90.31.15:4704 , piece_id=Q5OMCQTADPUW6BRQJXZ52PAQEPQ7ZNF3Q3PTQZ3WQGM55VOKMMXA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683391198 (592h0m30.454032533s) [noise=true , address=77.90.31.21:4904 , piece_id=X66X7H3DPI3WCAJUIGXNAA3VHAM3FUMP4IYPZRSK3ZBGUSJVDJGQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683761526 (592h0m30.454402861s) [noise=true , address=77.90.31.11:4562 , piece_id=V3BRXEN4M5QUC6G4S6GRU3JL4D7AGYPW3KQPOEVSYOXW2FQIRHRQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683801033 (592h0m30.454442368s) [noise=true , address=79.127.205.233:4641 , piece_id=ZITXZYCAPCMYSQZNGMMGS4K3OAOLYTILFH5QKLCXHW6NJIGPACLA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684063464 (592h0m30.454704799s) [noise=true , address=45.91.95.4:4012 , piece_id=TYKHU53FWX7OATNYZ6SJZXI2HGNVCY777A7CVDFXJORVO5MNII4A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684360503 (592h0m30.455001838s) [noise=true , address=79.127.205.245:4224 , piece_id=HAJ52S6XX66XD3C6ZMU56YP5OQEEYDPRTSTVDBGXOXHAHAUY3N3A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684654846 (592h0m30.455296181s) [noise=true , address=79.127.220.101:4045 , piece_id=ZZOB4VJ2UG4SQ2H43EIZGO3FHIQOENHFWAPGLKFM3RTYLK2DEKPQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.685431742 (592h0m30.456073077s) [noise=true , address=79.127.201.215:4349 , piece_id=VRU2SIUMMBI5UGH77OEGC6NL4P3UBSKVAN7WH2TP5B5UQNBZMC5A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.685861425 (592h0m30.45650276s) [noise=true , address=79.127.219.48:4107 , piece_id=UJMGZGLBBUHNAUSZORMGK37U2GMWDHDUE43YAOF2YMALCH6H5WEA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686139176 (592h0m30.456780511s) [noise=true , address=77.90.31.17:4783 , piece_id=67OLYYUNANWGBRK5TQFCSFYA3YR6BKAP2U5TCXQDZV2PVC5I2APA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686424722 (592h0m30.457066057s) [noise=true , address=77.90.31.18:4830 , piece_id=6DACHMTF7HVRL66TNQW4E2FOYIGLXKIQCFPD5GWYPOJAMFPB32PA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686687836 (592h0m30.457329171s) [noise=true , address=77.90.31.18:4821 , piece_id=DAXP3Y4RD4TF34YCWYAAMUNXVA26LYDTNJ64YTQYJ5ZPQMPB5D5Q, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686944527 (592h0m30.457585862s) [noise=true , address=77.90.31.19:4850 , piece_id=NNE6FK2G3YOIR5F6X426L5P6LQ6ZLHBJHCOMDKBVSXP2BICK4WFQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.687208402 (592h0m30.457849737s) [noise=true , address=104.36.82.50:4019 , piece_id=SOWQ4LMLRKAKIMR7YJI62NEV5K3JU3EINVRRDBL7PBMQV4WE4CPQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688069810 (592h0m30.458711145s) [noise=true , address=77.90.31.14:4689 , piece_id=YBQH3RBNJG3YACTBUTGRTVVXCOTQMIN7AM7H6Y6BEKK34QYKYAZQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688391375 (592h0m30.45903271s) [noise=true , address=77.90.31.13:4635 , piece_id=LUOGOPKTRW3BJIWBVTVRGXRFMXMYASMOXCW254FZQPRJOCQPXNQA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688755091 (592h0m30.459396426s) [noise=true , address=45.91.116.12:4254 , piece_id=QPEMFLQOTSSROFEKSDKARMJ7R6BZGY44TXAOLAP35ZH73TCHBNLQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.689020598 (592h0m30.459661934s) [noise=true , address=79.127.205.234:4595 , piece_id=MK6APXSMEA2T2Q6BZQUXEAKETGRYSJCFYOSAR3WEOWWPBKCNVXHQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:56.791786484 (592h0m31.562427839s) [upload=473462]: Next returned nil segment Jul 18 01:10:56.791810180 (592h0m31.562451535s) [upload=473462]: Waiting for error group managing segments... Jul 18 01:10:57.072508846 (592h0m31.843150181s) [noise=true , address=79.127.205.233:4641 , piece_id=ZITXZYCAPCMYSQZNGMMGS4K3OAOLYTILFH5QKLCXHW6NJIGPACLA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.084313581 (592h0m31.854954917s) [noise=true , address=79.127.205.245:4224 , piece_id=HAJ52S6XX66XD3C6ZMU56YP5OQEEYDPRTSTVDBGXOXHAHAUY3N3A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.130829109 (592h0m31.901470444s) [noise=true , address=79.127.205.234:4595 , piece_id=MK6APXSMEA2T2Q6BZQUXEAKETGRYSJCFYOSAR3WEOWWPBKCNVXHQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.380019692 (592h0m32.150661027s) [noise=true , address=79.127.220.101:4045 , piece_id=ZZOB4VJ2UG4SQ2H43EIZGO3FHIQOENHFWAPGLKFM3RTYLK2DEKPQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.586116023 (592h0m32.356757358s) [noise=true , address=79.127.219.48:4107 , piece_id=UJMGZGLBBUHNAUSZORMGK37U2GMWDHDUE43YAOF2YMALCH6H5WEA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.590030633 (592h0m32.360671968s) [noise=true , address=79.127.201.215:4349 , piece_id=VRU2SIUMMBI5UGH77OEGC6NL4P3UBSKVAN7WH2TP5B5UQNBZMC5A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.644593087 (592h0m32.415234412s) [noise=true , address=45.91.95.4:4012 , piece_id=TYKHU53FWX7OATNYZ6SJZXI2HGNVCY777A7CVDFXJORVO5MNII4A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.711170438 (592h0m32.481811773s) [noise=true , address=104.36.82.50:4019 , piece_id=SOWQ4LMLRKAKIMR7YJI62NEV5K3JU3EINVRRDBL7PBMQV4WE4CPQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.815350524 (592h0m32.585991869s) [noise=true , address=45.91.116.12:4254 , piece_id=QPEMFLQOTSSROFEKSDKARMJ7R6BZGY44TXAOLAP35ZH73TCHBNLQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:11:54.506942141 (592h1m29.277583476s) [noise=true , address=77.90.31.13:4635 , piece_id=LUOGOPKTRW3BJIWBVTVRGXRFMXMYASMOXCW254FZQPRJOCQPXNQA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1hkgeCi3efNQFv37chVFEi4v2AyWFEwv9keyy5oz6RkJ4Ubunp, address:77.90.31.13:4635): protocol: rpc: dial tcp 77.90.31.13:4635: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.506990634 (592h1m29.277631959s) [noise=true , address=77.90.31.14:4689 , piece_id=YBQH3RBNJG3YACTBUTGRTVVXCOTQMIN7AM7H6Y6BEKK34QYKYAZQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1bzvSPximqxjmXA4xtXZzD8Xyp27KURJ3oH1zCvbPmPwaLkuhM, address:77.90.31.14:4689): protocol: rpc: dial tcp 77.90.31.14:4689: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507021904 (592h1m29.277663239s) [noise=true , address=77.90.31.19:4850 , piece_id=NNE6FK2G3YOIR5F6X426L5P6LQ6ZLHBJHCOMDKBVSXP2BICK4WFQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12oGsNdVAKAabvVuWFZ2mpydVYjzhJycig4cqdcE5oAS2YsHmYv, address:77.90.31.19:4850): protocol: rpc: dial tcp 77.90.31.19:4850: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507055037 (592h1m29.277696372s) [noise=true , address=77.90.31.18:4821 , piece_id=DAXP3Y4RD4TF34YCWYAAMUNXVA26LYDTNJ64YTQYJ5ZPQMPB5D5Q, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12dSS7NcpaDB98oZdBM6vFe56mejA8fJZKA5YvERXTD9qqXVPdH, address:77.90.31.18:4821): protocol: rpc: dial tcp 77.90.31.18:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507085656 (592h1m29.277726992s) [noise=true , address=77.90.31.18:4830 , piece_id=6DACHMTF7HVRL66TNQW4E2FOYIGLXKIQCFPD5GWYPOJAMFPB32PA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz, address:77.90.31.18:4830): protocol: write tcp 188.166.225.170:42120->77.90.31.18:4830: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507153856 (592h1m29.277795192s) [noise=true , address=77.90.31.15:4704 , piece_id=Q5OMCQTADPUW6BRQJXZ52PAQEPQ7ZNF3Q3PTQZ3WQGM55VOKMMXA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, address:77.90.31.15:4704): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507228530 (592h1m29.277869865s) [noise=true , address=77.90.31.17:4783 , piece_id=67OLYYUNANWGBRK5TQFCSFYA3YR6BKAP2U5TCXQDZV2PVC5I2APA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507325304 (592h1m29.277966639s) [noise=true , address=77.90.31.21:4904 , piece_id=X66X7H3DPI3WCAJUIGXNAA3VHAM3FUMP4IYPZRSK3ZBGUSJVDJGQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12eUC555Lzo37B5FafAFMfXjSkfm1zv1pRZmH71jf4xpdCWH6kG, address:77.90.31.21:4904): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507433642 (592h1m29.278074977s) [noise=true , address=77.90.31.11:4562 , piece_id=V3BRXEN4M5QUC6G4S6GRU3JL4D7AGYPW3KQPOEVSYOXW2FQIRHRQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1SXQ4wGnLgKifp7pWWtKGtCom5K7fGceJHy2SyLGvUemZx4H4r, address:77.90.31.11:4562): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507448881 (592h1m29.278090216s) [seg_pos={0 0}, upload=473462]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Jul 18 01:11:54.756684719 (592h1m29.527326054s) [upload=473462]: Done uploading segments... Jul 18 01:11:54.756710668 (592h1m29.527352004s) [upload=473462]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 5.662969304s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 5.679550597s FAIL (2025-07-18 01:54:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== ress=77.90.31.15:4704 , piece_id=Q5OMCQTADPUW6BRQJXZ52PAQEPQ7ZNF3Q3PTQZ3WQGM55VOKMMXA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683391198 (592h0m30.454032533s) [noise=true , address=77.90.31.21:4904 , piece_id=X66X7H3DPI3WCAJUIGXNAA3VHAM3FUMP4IYPZRSK3ZBGUSJVDJGQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683761526 (592h0m30.454402861s) [noise=true , address=77.90.31.11:4562 , piece_id=V3BRXEN4M5QUC6G4S6GRU3JL4D7AGYPW3KQPOEVSYOXW2FQIRHRQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683801033 (592h0m30.454442368s) [noise=true , address=79.127.205.233:4641 , piece_id=ZITXZYCAPCMYSQZNGMMGS4K3OAOLYTILFH5QKLCXHW6NJIGPACLA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684063464 (592h0m30.454704799s) [noise=true , address=45.91.95.4:4012 , piece_id=TYKHU53FWX7OATNYZ6SJZXI2HGNVCY777A7CVDFXJORVO5MNII4A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684360503 (592h0m30.455001838s) [noise=true , address=79.127.205.245:4224 , piece_id=HAJ52S6XX66XD3C6ZMU56YP5OQEEYDPRTSTVDBGXOXHAHAUY3N3A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684654846 (592h0m30.455296181s) [noise=true , address=79.127.220.101:4045 , piece_id=ZZOB4VJ2UG4SQ2H43EIZGO3FHIQOENHFWAPGLKFM3RTYLK2DEKPQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.685431742 (592h0m30.456073077s) [noise=true , address=79.127.201.215:4349 , piece_id=VRU2SIUMMBI5UGH77OEGC6NL4P3UBSKVAN7WH2TP5B5UQNBZMC5A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.685861425 (592h0m30.45650276s) [noise=true , address=79.127.219.48:4107 , piece_id=UJMGZGLBBUHNAUSZORMGK37U2GMWDHDUE43YAOF2YMALCH6H5WEA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686139176 (592h0m30.456780511s) [noise=true , address=77.90.31.17:4783 , piece_id=67OLYYUNANWGBRK5TQFCSFYA3YR6BKAP2U5TCXQDZV2PVC5I2APA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686424722 (592h0m30.457066057s) [noise=true , address=77.90.31.18:4830 , piece_id=6DACHMTF7HVRL66TNQW4E2FOYIGLXKIQCFPD5GWYPOJAMFPB32PA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686687836 (592h0m30.457329171s) [noise=true , address=77.90.31.18:4821 , piece_id=DAXP3Y4RD4TF34YCWYAAMUNXVA26LYDTNJ64YTQYJ5ZPQMPB5D5Q, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686944527 (592h0m30.457585862s) [noise=true , address=77.90.31.19:4850 , piece_id=NNE6FK2G3YOIR5F6X426L5P6LQ6ZLHBJHCOMDKBVSXP2BICK4WFQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.687208402 (592h0m30.457849737s) [noise=true , address=104.36.82.50:4019 , piece_id=SOWQ4LMLRKAKIMR7YJI62NEV5K3JU3EINVRRDBL7PBMQV4WE4CPQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688069810 (592h0m30.458711145s) [noise=true , address=77.90.31.14:4689 , piece_id=YBQH3RBNJG3YACTBUTGRTVVXCOTQMIN7AM7H6Y6BEKK34QYKYAZQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688391375 (592h0m30.45903271s) [noise=true , address=77.90.31.13:4635 , piece_id=LUOGOPKTRW3BJIWBVTVRGXRFMXMYASMOXCW254FZQPRJOCQPXNQA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688755091 (592h0m30.459396426s) [noise=true , address=45.91.116.12:4254 , piece_id=QPEMFLQOTSSROFEKSDKARMJ7R6BZGY44TXAOLAP35ZH73TCHBNLQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.689020598 (592h0m30.459661934s) [noise=true , address=79.127.205.234:4595 , piece_id=MK6APXSMEA2T2Q6BZQUXEAKETGRYSJCFYOSAR3WEOWWPBKCNVXHQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:56.791786484 (592h0m31.562427839s) [upload=473462]: Next returned nil segment Jul 18 01:10:56.791810180 (592h0m31.562451535s) [upload=473462]: Waiting for error group managing segments... Jul 18 01:10:57.072508846 (592h0m31.843150181s) [noise=true , address=79.127.205.233:4641 , piece_id=ZITXZYCAPCMYSQZNGMMGS4K3OAOLYTILFH5QKLCXHW6NJIGPACLA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.084313581 (592h0m31.854954917s) [noise=true , address=79.127.205.245:4224 , piece_id=HAJ52S6XX66XD3C6ZMU56YP5OQEEYDPRTSTVDBGXOXHAHAUY3N3A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.130829109 (592h0m31.901470444s) [noise=true , address=79.127.205.234:4595 , piece_id=MK6APXSMEA2T2Q6BZQUXEAKETGRYSJCFYOSAR3WEOWWPBKCNVXHQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.380019692 (592h0m32.150661027s) [noise=true , address=79.127.220.101:4045 , piece_id=ZZOB4VJ2UG4SQ2H43EIZGO3FHIQOENHFWAPGLKFM3RTYLK2DEKPQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.586116023 (592h0m32.356757358s) [noise=true , address=79.127.219.48:4107 , piece_id=UJMGZGLBBUHNAUSZORMGK37U2GMWDHDUE43YAOF2YMALCH6H5WEA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.590030633 (592h0m32.360671968s) [noise=true , address=79.127.201.215:4349 , piece_id=VRU2SIUMMBI5UGH77OEGC6NL4P3UBSKVAN7WH2TP5B5UQNBZMC5A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.644593087 (592h0m32.415234412s) [noise=true , address=45.91.95.4:4012 , piece_id=TYKHU53FWX7OATNYZ6SJZXI2HGNVCY777A7CVDFXJORVO5MNII4A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.711170438 (592h0m32.481811773s) [noise=true , address=104.36.82.50:4019 , piece_id=SOWQ4LMLRKAKIMR7YJI62NEV5K3JU3EINVRRDBL7PBMQV4WE4CPQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.815350524 (592h0m32.585991869s) [noise=true , address=45.91.116.12:4254 , piece_id=QPEMFLQOTSSROFEKSDKARMJ7R6BZGY44TXAOLAP35ZH73TCHBNLQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:11:54.506942141 (592h1m29.277583476s) [noise=true , address=77.90.31.13:4635 , piece_id=LUOGOPKTRW3BJIWBVTVRGXRFMXMYASMOXCW254FZQPRJOCQPXNQA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1hkgeCi3efNQFv37chVFEi4v2AyWFEwv9keyy5oz6RkJ4Ubunp, address:77.90.31.13:4635): protocol: rpc: dial tcp 77.90.31.13:4635: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.506990634 (592h1m29.277631959s) [noise=true , address=77.90.31.14:4689 , piece_id=YBQH3RBNJG3YACTBUTGRTVVXCOTQMIN7AM7H6Y6BEKK34QYKYAZQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1bzvSPximqxjmXA4xtXZzD8Xyp27KURJ3oH1zCvbPmPwaLkuhM, address:77.90.31.14:4689): protocol: rpc: dial tcp 77.90.31.14:4689: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507021904 (592h1m29.277663239s) [noise=true , address=77.90.31.19:4850 , piece_id=NNE6FK2G3YOIR5F6X426L5P6LQ6ZLHBJHCOMDKBVSXP2BICK4WFQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12oGsNdVAKAabvVuWFZ2mpydVYjzhJycig4cqdcE5oAS2YsHmYv, address:77.90.31.19:4850): protocol: rpc: dial tcp 77.90.31.19:4850: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507055037 (592h1m29.277696372s) [noise=true , address=77.90.31.18:4821 , piece_id=DAXP3Y4RD4TF34YCWYAAMUNXVA26LYDTNJ64YTQYJ5ZPQMPB5D5Q, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12dSS7NcpaDB98oZdBM6vFe56mejA8fJZKA5YvERXTD9qqXVPdH, address:77.90.31.18:4821): protocol: rpc: dial tcp 77.90.31.18:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507085656 (592h1m29.277726992s) [noise=true , address=77.90.31.18:4830 , piece_id=6DACHMTF7HVRL66TNQW4E2FOYIGLXKIQCFPD5GWYPOJAMFPB32PA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz, address:77.90.31.18:4830): protocol: write tcp 188.166.225.170:42120->77.90.31.18:4830: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507153856 (592h1m29.277795192s) [noise=true , address=77.90.31.15:4704 , piece_id=Q5OMCQTADPUW6BRQJXZ52PAQEPQ7ZNF3Q3PTQZ3WQGM55VOKMMXA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, address:77.90.31.15:4704): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507228530 (592h1m29.277869865s) [noise=true , address=77.90.31.17:4783 , piece_id=67OLYYUNANWGBRK5TQFCSFYA3YR6BKAP2U5TCXQDZV2PVC5I2APA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507325304 (592h1m29.277966639s) [noise=true , address=77.90.31.21:4904 , piece_id=X66X7H3DPI3WCAJUIGXNAA3VHAM3FUMP4IYPZRSK3ZBGUSJVDJGQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12eUC555Lzo37B5FafAFMfXjSkfm1zv1pRZmH71jf4xpdCWH6kG, address:77.90.31.21:4904): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507433642 (592h1m29.278074977s) [noise=true , address=77.90.31.11:4562 , piece_id=V3BRXEN4M5QUC6G4S6GRU3JL4D7AGYPW3KQPOEVSYOXW2FQIRHRQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1SXQ4wGnLgKifp7pWWtKGtCom5K7fGceJHy2SyLGvUemZx4H4r, address:77.90.31.11:4562): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507448881 (592h1m29.278090216s) [seg_pos={0 0}, upload=473462]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Jul 18 01:11:54.756684719 (592h1m29.527326054s) [upload=473462]: Done uploading segments... Jul 18 01:11:54.756710668 (592h1m29.527352004s) [upload=473462]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 5.679581237s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 5.769028244s FAIL (2025-07-18 01:53:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== ress=77.90.31.15:4704 , piece_id=Q5OMCQTADPUW6BRQJXZ52PAQEPQ7ZNF3Q3PTQZ3WQGM55VOKMMXA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683391198 (592h0m30.454032533s) [noise=true , address=77.90.31.21:4904 , piece_id=X66X7H3DPI3WCAJUIGXNAA3VHAM3FUMP4IYPZRSK3ZBGUSJVDJGQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683761526 (592h0m30.454402861s) [noise=true , address=77.90.31.11:4562 , piece_id=V3BRXEN4M5QUC6G4S6GRU3JL4D7AGYPW3KQPOEVSYOXW2FQIRHRQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683801033 (592h0m30.454442368s) [noise=true , address=79.127.205.233:4641 , piece_id=ZITXZYCAPCMYSQZNGMMGS4K3OAOLYTILFH5QKLCXHW6NJIGPACLA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684063464 (592h0m30.454704799s) [noise=true , address=45.91.95.4:4012 , piece_id=TYKHU53FWX7OATNYZ6SJZXI2HGNVCY777A7CVDFXJORVO5MNII4A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684360503 (592h0m30.455001838s) [noise=true , address=79.127.205.245:4224 , piece_id=HAJ52S6XX66XD3C6ZMU56YP5OQEEYDPRTSTVDBGXOXHAHAUY3N3A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684654846 (592h0m30.455296181s) [noise=true , address=79.127.220.101:4045 , piece_id=ZZOB4VJ2UG4SQ2H43EIZGO3FHIQOENHFWAPGLKFM3RTYLK2DEKPQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.685431742 (592h0m30.456073077s) [noise=true , address=79.127.201.215:4349 , piece_id=VRU2SIUMMBI5UGH77OEGC6NL4P3UBSKVAN7WH2TP5B5UQNBZMC5A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.685861425 (592h0m30.45650276s) [noise=true , address=79.127.219.48:4107 , piece_id=UJMGZGLBBUHNAUSZORMGK37U2GMWDHDUE43YAOF2YMALCH6H5WEA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686139176 (592h0m30.456780511s) [noise=true , address=77.90.31.17:4783 , piece_id=67OLYYUNANWGBRK5TQFCSFYA3YR6BKAP2U5TCXQDZV2PVC5I2APA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686424722 (592h0m30.457066057s) [noise=true , address=77.90.31.18:4830 , piece_id=6DACHMTF7HVRL66TNQW4E2FOYIGLXKIQCFPD5GWYPOJAMFPB32PA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686687836 (592h0m30.457329171s) [noise=true , address=77.90.31.18:4821 , piece_id=DAXP3Y4RD4TF34YCWYAAMUNXVA26LYDTNJ64YTQYJ5ZPQMPB5D5Q, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686944527 (592h0m30.457585862s) [noise=true , address=77.90.31.19:4850 , piece_id=NNE6FK2G3YOIR5F6X426L5P6LQ6ZLHBJHCOMDKBVSXP2BICK4WFQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.687208402 (592h0m30.457849737s) [noise=true , address=104.36.82.50:4019 , piece_id=SOWQ4LMLRKAKIMR7YJI62NEV5K3JU3EINVRRDBL7PBMQV4WE4CPQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688069810 (592h0m30.458711145s) [noise=true , address=77.90.31.14:4689 , piece_id=YBQH3RBNJG3YACTBUTGRTVVXCOTQMIN7AM7H6Y6BEKK34QYKYAZQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688391375 (592h0m30.45903271s) [noise=true , address=77.90.31.13:4635 , piece_id=LUOGOPKTRW3BJIWBVTVRGXRFMXMYASMOXCW254FZQPRJOCQPXNQA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688755091 (592h0m30.459396426s) [noise=true , address=45.91.116.12:4254 , piece_id=QPEMFLQOTSSROFEKSDKARMJ7R6BZGY44TXAOLAP35ZH73TCHBNLQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.689020598 (592h0m30.459661934s) [noise=true , address=79.127.205.234:4595 , piece_id=MK6APXSMEA2T2Q6BZQUXEAKETGRYSJCFYOSAR3WEOWWPBKCNVXHQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:56.791786484 (592h0m31.562427839s) [upload=473462]: Next returned nil segment Jul 18 01:10:56.791810180 (592h0m31.562451535s) [upload=473462]: Waiting for error group managing segments... Jul 18 01:10:57.072508846 (592h0m31.843150181s) [noise=true , address=79.127.205.233:4641 , piece_id=ZITXZYCAPCMYSQZNGMMGS4K3OAOLYTILFH5QKLCXHW6NJIGPACLA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.084313581 (592h0m31.854954917s) [noise=true , address=79.127.205.245:4224 , piece_id=HAJ52S6XX66XD3C6ZMU56YP5OQEEYDPRTSTVDBGXOXHAHAUY3N3A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.130829109 (592h0m31.901470444s) [noise=true , address=79.127.205.234:4595 , piece_id=MK6APXSMEA2T2Q6BZQUXEAKETGRYSJCFYOSAR3WEOWWPBKCNVXHQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.380019692 (592h0m32.150661027s) [noise=true , address=79.127.220.101:4045 , piece_id=ZZOB4VJ2UG4SQ2H43EIZGO3FHIQOENHFWAPGLKFM3RTYLK2DEKPQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.586116023 (592h0m32.356757358s) [noise=true , address=79.127.219.48:4107 , piece_id=UJMGZGLBBUHNAUSZORMGK37U2GMWDHDUE43YAOF2YMALCH6H5WEA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.590030633 (592h0m32.360671968s) [noise=true , address=79.127.201.215:4349 , piece_id=VRU2SIUMMBI5UGH77OEGC6NL4P3UBSKVAN7WH2TP5B5UQNBZMC5A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.644593087 (592h0m32.415234412s) [noise=true , address=45.91.95.4:4012 , piece_id=TYKHU53FWX7OATNYZ6SJZXI2HGNVCY777A7CVDFXJORVO5MNII4A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.711170438 (592h0m32.481811773s) [noise=true , address=104.36.82.50:4019 , piece_id=SOWQ4LMLRKAKIMR7YJI62NEV5K3JU3EINVRRDBL7PBMQV4WE4CPQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.815350524 (592h0m32.585991869s) [noise=true , address=45.91.116.12:4254 , piece_id=QPEMFLQOTSSROFEKSDKARMJ7R6BZGY44TXAOLAP35ZH73TCHBNLQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:11:54.506942141 (592h1m29.277583476s) [noise=true , address=77.90.31.13:4635 , piece_id=LUOGOPKTRW3BJIWBVTVRGXRFMXMYASMOXCW254FZQPRJOCQPXNQA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1hkgeCi3efNQFv37chVFEi4v2AyWFEwv9keyy5oz6RkJ4Ubunp, address:77.90.31.13:4635): protocol: rpc: dial tcp 77.90.31.13:4635: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.506990634 (592h1m29.277631959s) [noise=true , address=77.90.31.14:4689 , piece_id=YBQH3RBNJG3YACTBUTGRTVVXCOTQMIN7AM7H6Y6BEKK34QYKYAZQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1bzvSPximqxjmXA4xtXZzD8Xyp27KURJ3oH1zCvbPmPwaLkuhM, address:77.90.31.14:4689): protocol: rpc: dial tcp 77.90.31.14:4689: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507021904 (592h1m29.277663239s) [noise=true , address=77.90.31.19:4850 , piece_id=NNE6FK2G3YOIR5F6X426L5P6LQ6ZLHBJHCOMDKBVSXP2BICK4WFQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12oGsNdVAKAabvVuWFZ2mpydVYjzhJycig4cqdcE5oAS2YsHmYv, address:77.90.31.19:4850): protocol: rpc: dial tcp 77.90.31.19:4850: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507055037 (592h1m29.277696372s) [noise=true , address=77.90.31.18:4821 , piece_id=DAXP3Y4RD4TF34YCWYAAMUNXVA26LYDTNJ64YTQYJ5ZPQMPB5D5Q, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12dSS7NcpaDB98oZdBM6vFe56mejA8fJZKA5YvERXTD9qqXVPdH, address:77.90.31.18:4821): protocol: rpc: dial tcp 77.90.31.18:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507085656 (592h1m29.277726992s) [noise=true , address=77.90.31.18:4830 , piece_id=6DACHMTF7HVRL66TNQW4E2FOYIGLXKIQCFPD5GWYPOJAMFPB32PA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz, address:77.90.31.18:4830): protocol: write tcp 188.166.225.170:42120->77.90.31.18:4830: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507153856 (592h1m29.277795192s) [noise=true , address=77.90.31.15:4704 , piece_id=Q5OMCQTADPUW6BRQJXZ52PAQEPQ7ZNF3Q3PTQZ3WQGM55VOKMMXA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, address:77.90.31.15:4704): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507228530 (592h1m29.277869865s) [noise=true , address=77.90.31.17:4783 , piece_id=67OLYYUNANWGBRK5TQFCSFYA3YR6BKAP2U5TCXQDZV2PVC5I2APA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507325304 (592h1m29.277966639s) [noise=true , address=77.90.31.21:4904 , piece_id=X66X7H3DPI3WCAJUIGXNAA3VHAM3FUMP4IYPZRSK3ZBGUSJVDJGQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12eUC555Lzo37B5FafAFMfXjSkfm1zv1pRZmH71jf4xpdCWH6kG, address:77.90.31.21:4904): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507433642 (592h1m29.278074977s) [noise=true , address=77.90.31.11:4562 , piece_id=V3BRXEN4M5QUC6G4S6GRU3JL4D7AGYPW3KQPOEVSYOXW2FQIRHRQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1SXQ4wGnLgKifp7pWWtKGtCom5K7fGceJHy2SyLGvUemZx4H4r, address:77.90.31.11:4562): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507448881 (592h1m29.278090216s) [seg_pos={0 0}, upload=473462]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Jul 18 01:11:54.756684719 (592h1m29.527326054s) [upload=473462]: Done uploading segments... Jul 18 01:11:54.756710668 (592h1m29.527352004s) [upload=473462]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 5.769112645s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.860725579s FAIL (2025-07-18 01:52:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== T467KJ2SHEV5EJW2LJCYJSSHAG2CJIU2A, seg_pos={0 0}, upload=473444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:43510->77.90.31.16:4733: use of closed network connection; context canceled Jul 18 01:09:36.439338648 (591h59m11.209979983s) [seg_pos={0 0}, upload=473444]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 01:09:36.439407128 (591h59m11.210048464s) [upload=473444]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 01:09:36.439411076 (591h59m11.210052411s) [upload=473444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 01:09:36.756883131 (591h59m11.527524466s) [upload=473444]: Done uploading segments... Jul 18 01:09:36.756897708 (591h59m11.527539044s) [upload=473444]: Upload finished. err: <nil> Jul 18 01:10:39.025598816 (592h0m13.796240151s) [upload=473459]: Starting upload Jul 18 01:10:39.025658440 (592h0m13.796299765s) [upload=473459]: Done starting upload Jul 18 01:10:39.025759353 (592h0m13.796400688s) [upload=473459]: Uploading segments... Jul 18 01:10:39.025907185 (592h0m13.79654852s) [upload=473459]: Got next segment. Inline: false Jul 18 01:10:39.025950799 (592h0m13.796592124s) [upload=473459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 01:10:39.025957632 (592h0m13.796598967s) [upload=473459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 01:10:39.286495526 (592h0m14.057136861s) [seg_pos={0 0}, upload=473459]: Begin upload segment... Jul 18 01:10:39.286547334 (592h0m14.057188699s) [seg_pos={0 0}, upload=473459]: Done begin upload segment. Jul 18 01:10:39.286596759 (592h0m14.057238094s) [noise=true , address=104.36.82.13:4187 , piece_id=7JSX6JTF7PFLA5EJJKP5ERYMZ3WUZMBXKP5SEIOJQEJ64YCAWLNQ, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.286620685 (592h0m14.05726202s) [noise=true , address=104.36.82.5:4840 , piece_id=E5XS5KTCG5S3ELBLWWY5MSHSEHI6YF3C45KFZASYI55U2MAIJQNQ, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.287116924 (592h0m14.057758259s) [noise=true , address=104.36.82.2:4730 , piece_id=IQPV5SIE4KNSQI4LFTT7IR3IPF5OLW5G67NUSMF23O2GNR5GAWUQ, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.287176458 (592h0m14.057817793s) [noise=true , address=45.91.116.12:4226 , piece_id=5ZXQMWANA43PWXPQWE4LL5OCBEKETXV2EZC4VPRPF63N52PG7X4A, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.287488926 (592h0m14.058130261s) [noise=true , address=104.36.82.13:4185 , piece_id=OOVUZS6RGGFLT5RS47MLGNSI6O4QGTCAIFTVISY3YSRULSDDMDSQ, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.287772308 (592h0m14.058413643s) [noise=true , address=77.90.31.13:4627 , piece_id=DP5K3YEDPDYIFDVARNAW3FM2LNQNZBCMXHPSQIOGWYKXNFKD6DSA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.287863583 (592h0m14.058504918s) [noise=true , address=104.36.82.16:4295 , piece_id=N2YQBIMKQ7DEOWFMXWTBCNUMWXYKO2E2BEAKNMZYGUZ5N4BRUNTA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.288079506 (592h0m14.058720841s) [noise=true , address=77.90.31.15:4699 , piece_id=BFA5C3AXYQSLHJBMR4PSU3XNFBK3IJHADGJEHOWK34FCAUL67KFA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.288237839 (592h0m14.058879184s) [noise=true , address=77.90.31.19:4842 , piece_id=AX6OKKHUNOLSUPVQE3JXQKSGEN54AGDIAR6JGAYOPMGE2NTEVXXA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.288418594 (592h0m14.059059929s) [noise=true , address=45.91.116.4:4937 , piece_id=MHBQFHGDBWQ4NGAQNIY5NVYAT2ELG2BAVIDSRWZRSORFR35WVUFQ, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.288629087 (592h0m14.059270422s) [noise=true , address=45.91.116.10:4082 , piece_id=E5X3OLJ7LZXI77WBGWBRNMFRNS6NGX5PFVPUOWPBJVYTGWDWB4YA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.289488541 (592h0m14.060129877s) [noise=true , address=77.90.31.19:4841 , piece_id=VXP6EFC7M6GEHWE4556WCZ7UYNWIQOXLQ3DJR4YU66IRQY2MBAIA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.289738971 (592h0m14.060380306s) [noise=true , address=104.36.82.15:4259 , piece_id=RC3W63JUQD22JS4YV3P3FX4THBBPTVN4KMPWMSOZSEMKBPVABC5Q, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.289989741 (592h0m14.060631086s) [noise=true , address=45.91.116.9:4262 , piece_id=P52ZRQP7U767JA7SSYWS6WPSN77X2O3AC6MNKJCESIMDC5O2XJGA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.290216545 (592h0m14.06085788s) [noise=true , address=45.91.116.8:4047 , piece_id=7E6K2AWD3YANEN2WYD5CPT3U3VBWA6VHNVDF6DC5NJUGQHP3A3NA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.297416701 (592h0m14.068058036s) [noise=true , address=45.91.116.11:4118 , piece_id=VLA6S3CZOJZQSUYUIIE62FHEJDUEH35S2UP6CJOXFK7ATJCU2TVQ, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:41.030217031 (592h0m15.800858356s) [upload=473459]: Next returned nil segment Jul 18 01:10:41.030234745 (592h0m15.80087608s) [upload=473459]: Waiting for error group managing segments... Jul 18 01:10:41.308832612 (592h0m16.079473947s) [noise=true , address=45.91.116.8:4047 , piece_id=7E6K2AWD3YANEN2WYD5CPT3U3VBWA6VHNVDF6DC5NJUGQHP3A3NA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.358887261 (592h0m16.129528596s) [noise=true , address=104.36.82.16:4295 , piece_id=N2YQBIMKQ7DEOWFMXWTBCNUMWXYKO2E2BEAKNMZYGUZ5N4BRUNTA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.365986644 (592h0m16.136627979s) [noise=true , address=104.36.82.13:4185 , piece_id=OOVUZS6RGGFLT5RS47MLGNSI6O4QGTCAIFTVISY3YSRULSDDMDSQ, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.375224368 (592h0m16.145865704s) [noise=true , address=45.91.116.4:4937 , piece_id=MHBQFHGDBWQ4NGAQNIY5NVYAT2ELG2BAVIDSRWZRSORFR35WVUFQ, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.419739118 (592h0m16.190380453s) [noise=true , address=104.36.82.13:4187 , piece_id=7JSX6JTF7PFLA5EJJKP5ERYMZ3WUZMBXKP5SEIOJQEJ64YCAWLNQ, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.430360108 (592h0m16.201001444s) [noise=true , address=45.91.116.9:4262 , piece_id=P52ZRQP7U767JA7SSYWS6WPSN77X2O3AC6MNKJCESIMDC5O2XJGA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.450861434 (592h0m16.221502769s) [noise=true , address=104.36.82.5:4840 , piece_id=E5XS5KTCG5S3ELBLWWY5MSHSEHI6YF3C45KFZASYI55U2MAIJQNQ, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.468109615 (592h0m16.23875095s) [noise=true , address=45.91.116.10:4082 , piece_id=E5X3OLJ7LZXI77WBGWBRNMFRNS6NGX5PFVPUOWPBJVYTGWDWB4YA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.468727326 (592h0m16.239368661s) [noise=true , address=45.91.116.11:4118 , piece_id=VLA6S3CZOJZQSUYUIIE62FHEJDUEH35S2UP6CJOXFK7ATJCU2TVQ, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.516013518 (592h0m16.286654863s) [noise=true , address=104.36.82.15:4259 , piece_id=RC3W63JUQD22JS4YV3P3FX4THBBPTVN4KMPWMSOZSEMKBPVABC5Q, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.522587155 (592h0m16.29322849s) [noise=true , address=104.36.82.2:4730 , piece_id=IQPV5SIE4KNSQI4LFTT7IR3IPF5OLW5G67NUSMF23O2GNR5GAWUQ, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.553218502 (592h0m16.323859837s) [noise=true , address=45.91.116.12:4226 , piece_id=5ZXQMWANA43PWXPQWE4LL5OCBEKETXV2EZC4VPRPF63N52PG7X4A, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:11:08.331622589 (592h0m43.102263924s) [noise=true , address=77.90.31.15:4699 , piece_id=BFA5C3AXYQSLHJBMR4PSU3XNFBK3IJHADGJEHOWK34FCAUL67KFA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:38902->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:08.331668216 (592h0m43.102309551s) [noise=true , address=77.90.31.13:4627 , piece_id=DP5K3YEDPDYIFDVARNAW3FM2LNQNZBCMXHPSQIOGWYKXNFKD6DSA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:08.331710026 (592h0m43.102351361s) [noise=true , address=77.90.31.19:4841 , piece_id=VXP6EFC7M6GEHWE4556WCZ7UYNWIQOXLQ3DJR4YU66IRQY2MBAIA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:60806->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:08.331772906 (592h0m43.102414241s) [noise=true , address=77.90.31.19:4842 , piece_id=AX6OKKHUNOLSUPVQE3JXQKSGEN54AGDIAR6JGAYOPMGE2NTEVXXA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:08.331804857 (592h0m43.102446182s) [seg_pos={0 0}, upload=473459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 01:11:08.598012820 (592h0m43.368654156s) [upload=473459]: Done uploading segments... Jul 18 01:11:08.598041896 (592h0m43.368683231s) [upload=473459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.860749325s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 5.976963705s FAIL (2025-07-18 01:52:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== ress=77.90.31.15:4704 , piece_id=Q5OMCQTADPUW6BRQJXZ52PAQEPQ7ZNF3Q3PTQZ3WQGM55VOKMMXA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683391198 (592h0m30.454032533s) [noise=true , address=77.90.31.21:4904 , piece_id=X66X7H3DPI3WCAJUIGXNAA3VHAM3FUMP4IYPZRSK3ZBGUSJVDJGQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683761526 (592h0m30.454402861s) [noise=true , address=77.90.31.11:4562 , piece_id=V3BRXEN4M5QUC6G4S6GRU3JL4D7AGYPW3KQPOEVSYOXW2FQIRHRQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683801033 (592h0m30.454442368s) [noise=true , address=79.127.205.233:4641 , piece_id=ZITXZYCAPCMYSQZNGMMGS4K3OAOLYTILFH5QKLCXHW6NJIGPACLA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684063464 (592h0m30.454704799s) [noise=true , address=45.91.95.4:4012 , piece_id=TYKHU53FWX7OATNYZ6SJZXI2HGNVCY777A7CVDFXJORVO5MNII4A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684360503 (592h0m30.455001838s) [noise=true , address=79.127.205.245:4224 , piece_id=HAJ52S6XX66XD3C6ZMU56YP5OQEEYDPRTSTVDBGXOXHAHAUY3N3A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684654846 (592h0m30.455296181s) [noise=true , address=79.127.220.101:4045 , piece_id=ZZOB4VJ2UG4SQ2H43EIZGO3FHIQOENHFWAPGLKFM3RTYLK2DEKPQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.685431742 (592h0m30.456073077s) [noise=true , address=79.127.201.215:4349 , piece_id=VRU2SIUMMBI5UGH77OEGC6NL4P3UBSKVAN7WH2TP5B5UQNBZMC5A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.685861425 (592h0m30.45650276s) [noise=true , address=79.127.219.48:4107 , piece_id=UJMGZGLBBUHNAUSZORMGK37U2GMWDHDUE43YAOF2YMALCH6H5WEA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686139176 (592h0m30.456780511s) [noise=true , address=77.90.31.17:4783 , piece_id=67OLYYUNANWGBRK5TQFCSFYA3YR6BKAP2U5TCXQDZV2PVC5I2APA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686424722 (592h0m30.457066057s) [noise=true , address=77.90.31.18:4830 , piece_id=6DACHMTF7HVRL66TNQW4E2FOYIGLXKIQCFPD5GWYPOJAMFPB32PA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686687836 (592h0m30.457329171s) [noise=true , address=77.90.31.18:4821 , piece_id=DAXP3Y4RD4TF34YCWYAAMUNXVA26LYDTNJ64YTQYJ5ZPQMPB5D5Q, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686944527 (592h0m30.457585862s) [noise=true , address=77.90.31.19:4850 , piece_id=NNE6FK2G3YOIR5F6X426L5P6LQ6ZLHBJHCOMDKBVSXP2BICK4WFQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.687208402 (592h0m30.457849737s) [noise=true , address=104.36.82.50:4019 , piece_id=SOWQ4LMLRKAKIMR7YJI62NEV5K3JU3EINVRRDBL7PBMQV4WE4CPQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688069810 (592h0m30.458711145s) [noise=true , address=77.90.31.14:4689 , piece_id=YBQH3RBNJG3YACTBUTGRTVVXCOTQMIN7AM7H6Y6BEKK34QYKYAZQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688391375 (592h0m30.45903271s) [noise=true , address=77.90.31.13:4635 , piece_id=LUOGOPKTRW3BJIWBVTVRGXRFMXMYASMOXCW254FZQPRJOCQPXNQA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688755091 (592h0m30.459396426s) [noise=true , address=45.91.116.12:4254 , piece_id=QPEMFLQOTSSROFEKSDKARMJ7R6BZGY44TXAOLAP35ZH73TCHBNLQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.689020598 (592h0m30.459661934s) [noise=true , address=79.127.205.234:4595 , piece_id=MK6APXSMEA2T2Q6BZQUXEAKETGRYSJCFYOSAR3WEOWWPBKCNVXHQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:56.791786484 (592h0m31.562427839s) [upload=473462]: Next returned nil segment Jul 18 01:10:56.791810180 (592h0m31.562451535s) [upload=473462]: Waiting for error group managing segments... Jul 18 01:10:57.072508846 (592h0m31.843150181s) [noise=true , address=79.127.205.233:4641 , piece_id=ZITXZYCAPCMYSQZNGMMGS4K3OAOLYTILFH5QKLCXHW6NJIGPACLA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.084313581 (592h0m31.854954917s) [noise=true , address=79.127.205.245:4224 , piece_id=HAJ52S6XX66XD3C6ZMU56YP5OQEEYDPRTSTVDBGXOXHAHAUY3N3A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.130829109 (592h0m31.901470444s) [noise=true , address=79.127.205.234:4595 , piece_id=MK6APXSMEA2T2Q6BZQUXEAKETGRYSJCFYOSAR3WEOWWPBKCNVXHQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.380019692 (592h0m32.150661027s) [noise=true , address=79.127.220.101:4045 , piece_id=ZZOB4VJ2UG4SQ2H43EIZGO3FHIQOENHFWAPGLKFM3RTYLK2DEKPQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.586116023 (592h0m32.356757358s) [noise=true , address=79.127.219.48:4107 , piece_id=UJMGZGLBBUHNAUSZORMGK37U2GMWDHDUE43YAOF2YMALCH6H5WEA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.590030633 (592h0m32.360671968s) [noise=true , address=79.127.201.215:4349 , piece_id=VRU2SIUMMBI5UGH77OEGC6NL4P3UBSKVAN7WH2TP5B5UQNBZMC5A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.644593087 (592h0m32.415234412s) [noise=true , address=45.91.95.4:4012 , piece_id=TYKHU53FWX7OATNYZ6SJZXI2HGNVCY777A7CVDFXJORVO5MNII4A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.711170438 (592h0m32.481811773s) [noise=true , address=104.36.82.50:4019 , piece_id=SOWQ4LMLRKAKIMR7YJI62NEV5K3JU3EINVRRDBL7PBMQV4WE4CPQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.815350524 (592h0m32.585991869s) [noise=true , address=45.91.116.12:4254 , piece_id=QPEMFLQOTSSROFEKSDKARMJ7R6BZGY44TXAOLAP35ZH73TCHBNLQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:11:54.506942141 (592h1m29.277583476s) [noise=true , address=77.90.31.13:4635 , piece_id=LUOGOPKTRW3BJIWBVTVRGXRFMXMYASMOXCW254FZQPRJOCQPXNQA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1hkgeCi3efNQFv37chVFEi4v2AyWFEwv9keyy5oz6RkJ4Ubunp, address:77.90.31.13:4635): protocol: rpc: dial tcp 77.90.31.13:4635: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.506990634 (592h1m29.277631959s) [noise=true , address=77.90.31.14:4689 , piece_id=YBQH3RBNJG3YACTBUTGRTVVXCOTQMIN7AM7H6Y6BEKK34QYKYAZQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1bzvSPximqxjmXA4xtXZzD8Xyp27KURJ3oH1zCvbPmPwaLkuhM, address:77.90.31.14:4689): protocol: rpc: dial tcp 77.90.31.14:4689: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507021904 (592h1m29.277663239s) [noise=true , address=77.90.31.19:4850 , piece_id=NNE6FK2G3YOIR5F6X426L5P6LQ6ZLHBJHCOMDKBVSXP2BICK4WFQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12oGsNdVAKAabvVuWFZ2mpydVYjzhJycig4cqdcE5oAS2YsHmYv, address:77.90.31.19:4850): protocol: rpc: dial tcp 77.90.31.19:4850: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507055037 (592h1m29.277696372s) [noise=true , address=77.90.31.18:4821 , piece_id=DAXP3Y4RD4TF34YCWYAAMUNXVA26LYDTNJ64YTQYJ5ZPQMPB5D5Q, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12dSS7NcpaDB98oZdBM6vFe56mejA8fJZKA5YvERXTD9qqXVPdH, address:77.90.31.18:4821): protocol: rpc: dial tcp 77.90.31.18:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507085656 (592h1m29.277726992s) [noise=true , address=77.90.31.18:4830 , piece_id=6DACHMTF7HVRL66TNQW4E2FOYIGLXKIQCFPD5GWYPOJAMFPB32PA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz, address:77.90.31.18:4830): protocol: write tcp 188.166.225.170:42120->77.90.31.18:4830: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507153856 (592h1m29.277795192s) [noise=true , address=77.90.31.15:4704 , piece_id=Q5OMCQTADPUW6BRQJXZ52PAQEPQ7ZNF3Q3PTQZ3WQGM55VOKMMXA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, address:77.90.31.15:4704): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507228530 (592h1m29.277869865s) [noise=true , address=77.90.31.17:4783 , piece_id=67OLYYUNANWGBRK5TQFCSFYA3YR6BKAP2U5TCXQDZV2PVC5I2APA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507325304 (592h1m29.277966639s) [noise=true , address=77.90.31.21:4904 , piece_id=X66X7H3DPI3WCAJUIGXNAA3VHAM3FUMP4IYPZRSK3ZBGUSJVDJGQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12eUC555Lzo37B5FafAFMfXjSkfm1zv1pRZmH71jf4xpdCWH6kG, address:77.90.31.21:4904): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507433642 (592h1m29.278074977s) [noise=true , address=77.90.31.11:4562 , piece_id=V3BRXEN4M5QUC6G4S6GRU3JL4D7AGYPW3KQPOEVSYOXW2FQIRHRQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1SXQ4wGnLgKifp7pWWtKGtCom5K7fGceJHy2SyLGvUemZx4H4r, address:77.90.31.11:4562): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507448881 (592h1m29.278090216s) [seg_pos={0 0}, upload=473462]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Jul 18 01:11:54.756684719 (592h1m29.527326054s) [upload=473462]: Done uploading segments... Jul 18 01:11:54.756710668 (592h1m29.527352004s) [upload=473462]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 5.976992039s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 6.182155096s FAIL (2025-07-18 01:52:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 9) ======LOGS====== ress=77.90.31.15:4704 , piece_id=Q5OMCQTADPUW6BRQJXZ52PAQEPQ7ZNF3Q3PTQZ3WQGM55VOKMMXA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683391198 (592h0m30.454032533s) [noise=true , address=77.90.31.21:4904 , piece_id=X66X7H3DPI3WCAJUIGXNAA3VHAM3FUMP4IYPZRSK3ZBGUSJVDJGQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683761526 (592h0m30.454402861s) [noise=true , address=77.90.31.11:4562 , piece_id=V3BRXEN4M5QUC6G4S6GRU3JL4D7AGYPW3KQPOEVSYOXW2FQIRHRQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.683801033 (592h0m30.454442368s) [noise=true , address=79.127.205.233:4641 , piece_id=ZITXZYCAPCMYSQZNGMMGS4K3OAOLYTILFH5QKLCXHW6NJIGPACLA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684063464 (592h0m30.454704799s) [noise=true , address=45.91.95.4:4012 , piece_id=TYKHU53FWX7OATNYZ6SJZXI2HGNVCY777A7CVDFXJORVO5MNII4A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684360503 (592h0m30.455001838s) [noise=true , address=79.127.205.245:4224 , piece_id=HAJ52S6XX66XD3C6ZMU56YP5OQEEYDPRTSTVDBGXOXHAHAUY3N3A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.684654846 (592h0m30.455296181s) [noise=true , address=79.127.220.101:4045 , piece_id=ZZOB4VJ2UG4SQ2H43EIZGO3FHIQOENHFWAPGLKFM3RTYLK2DEKPQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.685431742 (592h0m30.456073077s) [noise=true , address=79.127.201.215:4349 , piece_id=VRU2SIUMMBI5UGH77OEGC6NL4P3UBSKVAN7WH2TP5B5UQNBZMC5A, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.685861425 (592h0m30.45650276s) [noise=true , address=79.127.219.48:4107 , piece_id=UJMGZGLBBUHNAUSZORMGK37U2GMWDHDUE43YAOF2YMALCH6H5WEA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686139176 (592h0m30.456780511s) [noise=true , address=77.90.31.17:4783 , piece_id=67OLYYUNANWGBRK5TQFCSFYA3YR6BKAP2U5TCXQDZV2PVC5I2APA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686424722 (592h0m30.457066057s) [noise=true , address=77.90.31.18:4830 , piece_id=6DACHMTF7HVRL66TNQW4E2FOYIGLXKIQCFPD5GWYPOJAMFPB32PA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686687836 (592h0m30.457329171s) [noise=true , address=77.90.31.18:4821 , piece_id=DAXP3Y4RD4TF34YCWYAAMUNXVA26LYDTNJ64YTQYJ5ZPQMPB5D5Q, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.686944527 (592h0m30.457585862s) [noise=true , address=77.90.31.19:4850 , piece_id=NNE6FK2G3YOIR5F6X426L5P6LQ6ZLHBJHCOMDKBVSXP2BICK4WFQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.687208402 (592h0m30.457849737s) [noise=true , address=104.36.82.50:4019 , piece_id=SOWQ4LMLRKAKIMR7YJI62NEV5K3JU3EINVRRDBL7PBMQV4WE4CPQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688069810 (592h0m30.458711145s) [noise=true , address=77.90.31.14:4689 , piece_id=YBQH3RBNJG3YACTBUTGRTVVXCOTQMIN7AM7H6Y6BEKK34QYKYAZQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688391375 (592h0m30.45903271s) [noise=true , address=77.90.31.13:4635 , piece_id=LUOGOPKTRW3BJIWBVTVRGXRFMXMYASMOXCW254FZQPRJOCQPXNQA, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.688755091 (592h0m30.459396426s) [noise=true , address=45.91.116.12:4254 , piece_id=QPEMFLQOTSSROFEKSDKARMJ7R6BZGY44TXAOLAP35ZH73TCHBNLQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:55.689020598 (592h0m30.459661934s) [noise=true , address=79.127.205.234:4595 , piece_id=MK6APXSMEA2T2Q6BZQUXEAKETGRYSJCFYOSAR3WEOWWPBKCNVXHQ, seg_pos={0 0}, upload=473462]: Uploading piece... Jul 18 01:10:56.791786484 (592h0m31.562427839s) [upload=473462]: Next returned nil segment Jul 18 01:10:56.791810180 (592h0m31.562451535s) [upload=473462]: Waiting for error group managing segments... Jul 18 01:10:57.072508846 (592h0m31.843150181s) [noise=true , address=79.127.205.233:4641 , piece_id=ZITXZYCAPCMYSQZNGMMGS4K3OAOLYTILFH5QKLCXHW6NJIGPACLA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.084313581 (592h0m31.854954917s) [noise=true , address=79.127.205.245:4224 , piece_id=HAJ52S6XX66XD3C6ZMU56YP5OQEEYDPRTSTVDBGXOXHAHAUY3N3A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.130829109 (592h0m31.901470444s) [noise=true , address=79.127.205.234:4595 , piece_id=MK6APXSMEA2T2Q6BZQUXEAKETGRYSJCFYOSAR3WEOWWPBKCNVXHQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.380019692 (592h0m32.150661027s) [noise=true , address=79.127.220.101:4045 , piece_id=ZZOB4VJ2UG4SQ2H43EIZGO3FHIQOENHFWAPGLKFM3RTYLK2DEKPQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.586116023 (592h0m32.356757358s) [noise=true , address=79.127.219.48:4107 , piece_id=UJMGZGLBBUHNAUSZORMGK37U2GMWDHDUE43YAOF2YMALCH6H5WEA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.590030633 (592h0m32.360671968s) [noise=true , address=79.127.201.215:4349 , piece_id=VRU2SIUMMBI5UGH77OEGC6NL4P3UBSKVAN7WH2TP5B5UQNBZMC5A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.644593087 (592h0m32.415234412s) [noise=true , address=45.91.95.4:4012 , piece_id=TYKHU53FWX7OATNYZ6SJZXI2HGNVCY777A7CVDFXJORVO5MNII4A, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.711170438 (592h0m32.481811773s) [noise=true , address=104.36.82.50:4019 , piece_id=SOWQ4LMLRKAKIMR7YJI62NEV5K3JU3EINVRRDBL7PBMQV4WE4CPQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:10:57.815350524 (592h0m32.585991869s) [noise=true , address=45.91.116.12:4254 , piece_id=QPEMFLQOTSSROFEKSDKARMJ7R6BZGY44TXAOLAP35ZH73TCHBNLQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: <nil> Jul 18 01:11:54.506942141 (592h1m29.277583476s) [noise=true , address=77.90.31.13:4635 , piece_id=LUOGOPKTRW3BJIWBVTVRGXRFMXMYASMOXCW254FZQPRJOCQPXNQA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1hkgeCi3efNQFv37chVFEi4v2AyWFEwv9keyy5oz6RkJ4Ubunp, address:77.90.31.13:4635): protocol: rpc: dial tcp 77.90.31.13:4635: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.506990634 (592h1m29.277631959s) [noise=true , address=77.90.31.14:4689 , piece_id=YBQH3RBNJG3YACTBUTGRTVVXCOTQMIN7AM7H6Y6BEKK34QYKYAZQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1bzvSPximqxjmXA4xtXZzD8Xyp27KURJ3oH1zCvbPmPwaLkuhM, address:77.90.31.14:4689): protocol: rpc: dial tcp 77.90.31.14:4689: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507021904 (592h1m29.277663239s) [noise=true , address=77.90.31.19:4850 , piece_id=NNE6FK2G3YOIR5F6X426L5P6LQ6ZLHBJHCOMDKBVSXP2BICK4WFQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12oGsNdVAKAabvVuWFZ2mpydVYjzhJycig4cqdcE5oAS2YsHmYv, address:77.90.31.19:4850): protocol: rpc: dial tcp 77.90.31.19:4850: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507055037 (592h1m29.277696372s) [noise=true , address=77.90.31.18:4821 , piece_id=DAXP3Y4RD4TF34YCWYAAMUNXVA26LYDTNJ64YTQYJ5ZPQMPB5D5Q, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12dSS7NcpaDB98oZdBM6vFe56mejA8fJZKA5YvERXTD9qqXVPdH, address:77.90.31.18:4821): protocol: rpc: dial tcp 77.90.31.18:4821: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507085656 (592h1m29.277726992s) [noise=true , address=77.90.31.18:4830 , piece_id=6DACHMTF7HVRL66TNQW4E2FOYIGLXKIQCFPD5GWYPOJAMFPB32PA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12U2TBUpmxK1xeXXWHv1VNE8xgjQGU3MrcKsEmDRtPvpj2Tnddz, address:77.90.31.18:4830): protocol: write tcp 188.166.225.170:42120->77.90.31.18:4830: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:54.507153856 (592h1m29.277795192s) [noise=true , address=77.90.31.15:4704 , piece_id=Q5OMCQTADPUW6BRQJXZ52PAQEPQ7ZNF3Q3PTQZ3WQGM55VOKMMXA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:14tJRdZ1guwwsYwPf3LtENRWqYx2jBQpi7uAMMPaB32Gks9zmj, address:77.90.31.15:4704): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507228530 (592h1m29.277869865s) [noise=true , address=77.90.31.17:4783 , piece_id=67OLYYUNANWGBRK5TQFCSFYA3YR6BKAP2U5TCXQDZV2PVC5I2APA, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:155fvDXZdpQeHetitv17WCVVrP1bGswdoqJECQLSe66kXRhBPo, address:77.90.31.17:4783): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507325304 (592h1m29.277966639s) [noise=true , address=77.90.31.21:4904 , piece_id=X66X7H3DPI3WCAJUIGXNAA3VHAM3FUMP4IYPZRSK3ZBGUSJVDJGQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:12eUC555Lzo37B5FafAFMfXjSkfm1zv1pRZmH71jf4xpdCWH6kG, address:77.90.31.21:4904): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507433642 (592h1m29.278074977s) [noise=true , address=77.90.31.11:4562 , piece_id=V3BRXEN4M5QUC6G4S6GRU3JL4D7AGYPW3KQPOEVSYOXW2FQIRHRQ, seg_pos={0 0}, upload=473462]: Done uploading piece. err: ecclient: upload failed (node:1SXQ4wGnLgKifp7pWWtKGtCom5K7fGceJHy2SyLGvUemZx4H4r, address:77.90.31.11:4562): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:54.507448881 (592h1m29.278090216s) [seg_pos={0 0}, upload=473462]: Done waiting for segment. successful: 9 optimal: 14 errs: <nil> Jul 18 01:11:54.756684719 (592h1m29.527326054s) [upload=473462]: Done uploading segments... Jul 18 01:11:54.756710668 (592h1m29.527352004s) [upload=473462]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 9) Duration: 6.182179964s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.941480264s FAIL (2025-07-18 01:52:02 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== T467KJ2SHEV5EJW2LJCYJSSHAG2CJIU2A, seg_pos={0 0}, upload=473444]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: write tcp 188.166.225.170:43510->77.90.31.16:4733: use of closed network connection; context canceled Jul 18 01:09:36.439338648 (591h59m11.209979983s) [seg_pos={0 0}, upload=473444]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 18 01:09:36.439407128 (591h59m11.210048464s) [upload=473444]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 18 01:09:36.439411076 (591h59m11.210052411s) [upload=473444]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 18 01:09:36.756883131 (591h59m11.527524466s) [upload=473444]: Done uploading segments... Jul 18 01:09:36.756897708 (591h59m11.527539044s) [upload=473444]: Upload finished. err: <nil> Jul 18 01:10:39.025598816 (592h0m13.796240151s) [upload=473459]: Starting upload Jul 18 01:10:39.025658440 (592h0m13.796299765s) [upload=473459]: Done starting upload Jul 18 01:10:39.025759353 (592h0m13.796400688s) [upload=473459]: Uploading segments... Jul 18 01:10:39.025907185 (592h0m13.79654852s) [upload=473459]: Got next segment. Inline: false Jul 18 01:10:39.025950799 (592h0m13.796592124s) [upload=473459]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 18 01:10:39.025957632 (592h0m13.796598967s) [upload=473459]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 18 01:10:39.286495526 (592h0m14.057136861s) [seg_pos={0 0}, upload=473459]: Begin upload segment... Jul 18 01:10:39.286547334 (592h0m14.057188699s) [seg_pos={0 0}, upload=473459]: Done begin upload segment. Jul 18 01:10:39.286596759 (592h0m14.057238094s) [noise=true , address=104.36.82.13:4187 , piece_id=7JSX6JTF7PFLA5EJJKP5ERYMZ3WUZMBXKP5SEIOJQEJ64YCAWLNQ, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.286620685 (592h0m14.05726202s) [noise=true , address=104.36.82.5:4840 , piece_id=E5XS5KTCG5S3ELBLWWY5MSHSEHI6YF3C45KFZASYI55U2MAIJQNQ, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.287116924 (592h0m14.057758259s) [noise=true , address=104.36.82.2:4730 , piece_id=IQPV5SIE4KNSQI4LFTT7IR3IPF5OLW5G67NUSMF23O2GNR5GAWUQ, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.287176458 (592h0m14.057817793s) [noise=true , address=45.91.116.12:4226 , piece_id=5ZXQMWANA43PWXPQWE4LL5OCBEKETXV2EZC4VPRPF63N52PG7X4A, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.287488926 (592h0m14.058130261s) [noise=true , address=104.36.82.13:4185 , piece_id=OOVUZS6RGGFLT5RS47MLGNSI6O4QGTCAIFTVISY3YSRULSDDMDSQ, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.287772308 (592h0m14.058413643s) [noise=true , address=77.90.31.13:4627 , piece_id=DP5K3YEDPDYIFDVARNAW3FM2LNQNZBCMXHPSQIOGWYKXNFKD6DSA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.287863583 (592h0m14.058504918s) [noise=true , address=104.36.82.16:4295 , piece_id=N2YQBIMKQ7DEOWFMXWTBCNUMWXYKO2E2BEAKNMZYGUZ5N4BRUNTA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.288079506 (592h0m14.058720841s) [noise=true , address=77.90.31.15:4699 , piece_id=BFA5C3AXYQSLHJBMR4PSU3XNFBK3IJHADGJEHOWK34FCAUL67KFA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.288237839 (592h0m14.058879184s) [noise=true , address=77.90.31.19:4842 , piece_id=AX6OKKHUNOLSUPVQE3JXQKSGEN54AGDIAR6JGAYOPMGE2NTEVXXA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.288418594 (592h0m14.059059929s) [noise=true , address=45.91.116.4:4937 , piece_id=MHBQFHGDBWQ4NGAQNIY5NVYAT2ELG2BAVIDSRWZRSORFR35WVUFQ, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.288629087 (592h0m14.059270422s) [noise=true , address=45.91.116.10:4082 , piece_id=E5X3OLJ7LZXI77WBGWBRNMFRNS6NGX5PFVPUOWPBJVYTGWDWB4YA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.289488541 (592h0m14.060129877s) [noise=true , address=77.90.31.19:4841 , piece_id=VXP6EFC7M6GEHWE4556WCZ7UYNWIQOXLQ3DJR4YU66IRQY2MBAIA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.289738971 (592h0m14.060380306s) [noise=true , address=104.36.82.15:4259 , piece_id=RC3W63JUQD22JS4YV3P3FX4THBBPTVN4KMPWMSOZSEMKBPVABC5Q, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.289989741 (592h0m14.060631086s) [noise=true , address=45.91.116.9:4262 , piece_id=P52ZRQP7U767JA7SSYWS6WPSN77X2O3AC6MNKJCESIMDC5O2XJGA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.290216545 (592h0m14.06085788s) [noise=true , address=45.91.116.8:4047 , piece_id=7E6K2AWD3YANEN2WYD5CPT3U3VBWA6VHNVDF6DC5NJUGQHP3A3NA, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:39.297416701 (592h0m14.068058036s) [noise=true , address=45.91.116.11:4118 , piece_id=VLA6S3CZOJZQSUYUIIE62FHEJDUEH35S2UP6CJOXFK7ATJCU2TVQ, seg_pos={0 0}, upload=473459]: Uploading piece... Jul 18 01:10:41.030217031 (592h0m15.800858356s) [upload=473459]: Next returned nil segment Jul 18 01:10:41.030234745 (592h0m15.80087608s) [upload=473459]: Waiting for error group managing segments... Jul 18 01:10:41.308832612 (592h0m16.079473947s) [noise=true , address=45.91.116.8:4047 , piece_id=7E6K2AWD3YANEN2WYD5CPT3U3VBWA6VHNVDF6DC5NJUGQHP3A3NA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.358887261 (592h0m16.129528596s) [noise=true , address=104.36.82.16:4295 , piece_id=N2YQBIMKQ7DEOWFMXWTBCNUMWXYKO2E2BEAKNMZYGUZ5N4BRUNTA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.365986644 (592h0m16.136627979s) [noise=true , address=104.36.82.13:4185 , piece_id=OOVUZS6RGGFLT5RS47MLGNSI6O4QGTCAIFTVISY3YSRULSDDMDSQ, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.375224368 (592h0m16.145865704s) [noise=true , address=45.91.116.4:4937 , piece_id=MHBQFHGDBWQ4NGAQNIY5NVYAT2ELG2BAVIDSRWZRSORFR35WVUFQ, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.419739118 (592h0m16.190380453s) [noise=true , address=104.36.82.13:4187 , piece_id=7JSX6JTF7PFLA5EJJKP5ERYMZ3WUZMBXKP5SEIOJQEJ64YCAWLNQ, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.430360108 (592h0m16.201001444s) [noise=true , address=45.91.116.9:4262 , piece_id=P52ZRQP7U767JA7SSYWS6WPSN77X2O3AC6MNKJCESIMDC5O2XJGA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.450861434 (592h0m16.221502769s) [noise=true , address=104.36.82.5:4840 , piece_id=E5XS5KTCG5S3ELBLWWY5MSHSEHI6YF3C45KFZASYI55U2MAIJQNQ, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.468109615 (592h0m16.23875095s) [noise=true , address=45.91.116.10:4082 , piece_id=E5X3OLJ7LZXI77WBGWBRNMFRNS6NGX5PFVPUOWPBJVYTGWDWB4YA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.468727326 (592h0m16.239368661s) [noise=true , address=45.91.116.11:4118 , piece_id=VLA6S3CZOJZQSUYUIIE62FHEJDUEH35S2UP6CJOXFK7ATJCU2TVQ, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.516013518 (592h0m16.286654863s) [noise=true , address=104.36.82.15:4259 , piece_id=RC3W63JUQD22JS4YV3P3FX4THBBPTVN4KMPWMSOZSEMKBPVABC5Q, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.522587155 (592h0m16.29322849s) [noise=true , address=104.36.82.2:4730 , piece_id=IQPV5SIE4KNSQI4LFTT7IR3IPF5OLW5G67NUSMF23O2GNR5GAWUQ, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:10:41.553218502 (592h0m16.323859837s) [noise=true , address=45.91.116.12:4226 , piece_id=5ZXQMWANA43PWXPQWE4LL5OCBEKETXV2EZC4VPRPF63N52PG7X4A, seg_pos={0 0}, upload=473459]: Done uploading piece. err: <nil> Jul 18 01:11:08.331622589 (592h0m43.102263924s) [noise=true , address=77.90.31.15:4699 , piece_id=BFA5C3AXYQSLHJBMR4PSU3XNFBK3IJHADGJEHOWK34FCAUL67KFA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:38902->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:08.331668216 (592h0m43.102309551s) [noise=true , address=77.90.31.13:4627 , piece_id=DP5K3YEDPDYIFDVARNAW3FM2LNQNZBCMXHPSQIOGWYKXNFKD6DSA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: context deadline exceeded; context deadline exceeded Jul 18 01:11:08.331710026 (592h0m43.102351361s) [noise=true , address=77.90.31.19:4841 , piece_id=VXP6EFC7M6GEHWE4556WCZ7UYNWIQOXLQ3DJR4YU66IRQY2MBAIA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:60806->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:08.331772906 (592h0m43.102414241s) [noise=true , address=77.90.31.19:4842 , piece_id=AX6OKKHUNOLSUPVQE3JXQKSGEN54AGDIAR6JGAYOPMGE2NTEVXXA, seg_pos={0 0}, upload=473459]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 18 01:11:08.331804857 (592h0m43.102446182s) [seg_pos={0 0}, upload=473459]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Jul 18 01:11:08.598012820 (592h0m43.368654156s) [upload=473459]: Done uploading segments... Jul 18 01:11:08.598041896 (592h0m43.368683231s) [upload=473459]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.941502317s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)