Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..........................................XXX...............] duration: 5.834180221s FAIL (2025-07-19 19:57:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.834203114s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........................................XXXXXX..............] duration: 6.306689289s FAIL (2025-07-19 19:57:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.306719537s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...........................................XXX..............] duration: 5.85444585s FAIL (2025-07-19 19:56:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.854468974s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........................................XXXXXX.............] duration: 6.102817088s FAIL (2025-07-19 19:56:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.102842676s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [............................................XXX.............] duration: 5.78777292s FAIL (2025-07-19 19:55:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.787815281s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........................................XXXXXX............] duration: 5.925931713s FAIL (2025-07-19 19:55:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.925959476s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.............................................XXX............] duration: 5.624513264s FAIL (2025-07-19 19:55:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.624539815s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........................................XXXXXX...........] duration: 6.836457974s FAIL (2025-07-19 19:54:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.836486439s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..............................................XXX...........] duration: 6.764524619s FAIL (2025-07-19 19:54:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.764550359s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...............................................XXX..........] duration: 5.770898154s FAIL (2025-07-19 19:53:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.770932629s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............................................XXXXXX..........] duration: 5.784038734s FAIL (2025-07-19 19:53:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.784063312s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............................................XXXXXX.........] duration: 5.909804888s FAIL (2025-07-19 19:52:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.909844474s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [................................................XXX.........] duration: 5.821501665s FAIL (2025-07-19 19:52:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.821528526s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.................................................XXX........] duration: 5.626630334s FAIL (2025-07-19 19:51:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.626653247s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............................................XXXXXX........] duration: 7.209158428s FAIL (2025-07-19 19:51:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 7.209191262s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...............................................XXXXXX.......] duration: 6.650809744s FAIL (2025-07-19 19:51:25 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.650835523s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..................................................XXX.......] duration: 6.079805028s FAIL (2025-07-19 19:51:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.079842339s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [................................................XXXXXX......] duration: 5.971260638s FAIL (2025-07-19 19:51:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.971285786s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...................................................XXX......] duration: 5.868049967s FAIL (2025-07-19 19:50:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.868076437s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................................................XXXXXX.....] duration: 6.225290949s FAIL (2025-07-19 19:49:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.225315536s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [....................................................XXX.....] duration: 5.850446427s FAIL (2025-07-19 19:49:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.85047376s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..................................................XXXXXX....] duration: 5.86207144s FAIL (2025-07-19 19:49:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.862101827s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................XXX....] duration: 5.80490513s FAIL (2025-07-19 19:49:12 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.804929036s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...................................................XXXXXX...] duration: 5.946867187s FAIL (2025-07-19 19:48:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.946898258s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [......................................................XXX...] duration: 5.870110003s FAIL (2025-07-19 19:48:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.870132787s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....................................................XXXXXX..] duration: 6.068790571s FAIL (2025-07-19 19:47:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.068810729s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................XXX..] duration: 5.647576644s FAIL (2025-07-19 19:47:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.647604026s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................................................XXXXXX.] duration: 6.142495915s FAIL (2025-07-19 19:47:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.142521685s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XXX.] duration: 5.654933436s FAIL (2025-07-19 19:46:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.654950148s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 1m0.232975094s FAIL (2025-07-19 19:46:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.232986957s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration: 30.232293819s FAIL (2025-07-19 19:46:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.232317794s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 30.239869588s FAIL (2025-07-19 19:45:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 26s) [upload=16630]: Starting upload Jul 19 19:45:11.526300784 (20h54m17.898880139s) [upload=16630]: Done starting upload Jul 19 19:45:11.526374295 (20h54m17.89895365s) [upload=16630]: Uploading segments... Jul 19 19:45:11.526544621 (20h54m17.899123966s) [upload=16630]: Got next segment. Inline: false Jul 19 19:45:11.526571452 (20h54m17.899150807s) [upload=16630]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:11.526577012 (20h54m17.899156367s) [upload=16630]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:11.761315060 (20h54m18.133894415s) [seg_pos={0 0}, upload=16630]: Begin upload segment... Jul 19 19:45:11.761354145 (20h54m18.13393349s) [seg_pos={0 0}, upload=16630]: Done begin upload segment. Jul 19 19:45:11.761377780 (20h54m18.133957135s) [noise=true , address=77.90.31.12:4589 , piece_id=JNSJMAKN3KZFCA5VYO6EBKGZK3HFR7N5MSPTDAJLXE4LA5STMXEQ, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.761413719 (20h54m18.133993074s) [noise=true , address=104.36.82.11:4114 , piece_id=X3RYVF7Y74O7DK2BU5MFI6AAZH4666XNKPD23XNSXAEKJ2HSKHPQ, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.761715757 (20h54m18.134295112s) [noise=true , address=104.36.82.9:4007 , piece_id=72VJZHEK7QC4LXBNABD2ZP5BSC6RGUCPEAH3G54HCNEZVENWDUVQ, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.762031891 (20h54m18.134611246s) [noise=true , address=45.91.116.14:4191 , piece_id=I5KMV7RSA7FKOVMLUXKDI25SD2FIC5O6CLWWTUQMKRHHSVXRPB6A, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.762234149 (20h54m18.134813504s) [noise=true , address=104.36.82.13:4185 , piece_id=Y7Y2Z3QTOJVT2F6GOGETQNJRCQ5WA6ZXWC3E2QGPZNGDKXGKNYIQ, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.762287621 (20h54m18.134866976s) [noise=true , address=45.91.116.13:4154 , piece_id=ST4QCC4UTV24UXAAAXEPCFV2ISGY7NTQHNNEYDICFII3WT4NI55Q, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.762542929 (20h54m18.135122284s) [noise=true , address=104.36.82.16:4294 , piece_id=KVROAKA5NMPHID25WWD763DHMCB2WCVD2DXJBENLMWEIWDKWVEAA, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.762796915 (20h54m18.13537627s) [noise=true , address=45.91.116.14:4190 , piece_id=TK36JPUXM5YOKNJOHEFRVA7I4PYKFWN4P5IU4OO3QFECZWABNZUQ, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.762970037 (20h54m18.135549392s) [noise=true , address=104.36.82.10:4078 , piece_id=NAXJ6FRLBHPK362HP2WS27JJEE7JM3T43HWBPW57AQEU7R5O6XGA, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.763034891 (20h54m18.135614246s) [noise=true , address=104.36.82.8:4971 , piece_id=TLGYY4PJHTDOLV6ZR77UZJ5MHW3J5MGJEGWJP3Z5SXRQ7NYRJBVA, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.763205417 (20h54m18.135784772s) [noise=true , address=77.90.31.16:4735 , piece_id=4NWJMO5SSGY4OHFD5MFPPOWDZFNKIA5YXO5TUHGYYSQ5HE4BYN4Q, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.763292493 (20h54m18.135871848s) [noise=true , address=77.90.31.10:4517 , piece_id=C644MMDTSHF4N2EE6MFU4BBVXR63R63O62HH4K3DI6ND4ZPPB5MA, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.763492276 (20h54m18.136071631s) [noise=true , address=77.90.31.11:4554 , piece_id=N5U7LEC3JQZNQHU6JTHZMCXTAHBW6EEOZFOMINBGBBMF6SVXO6ZA, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.763519658 (20h54m18.136099013s) [noise=true , address=104.36.82.3:4767 , piece_id=L6MAL5Z4H6ARWVEAI2W7SEWNA44R2DN6W365ZNLDSF6YXSFKOIGQ, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.763709131 (20h54m18.136288486s) [noise=true , address=77.90.31.15:4699 , piece_id=3VXILCKSLCVXDD2O7EQFU5VZBS33G7TXHUKTWQ2PTWNKYHU7O5SQ, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:11.764014335 (20h54m18.13659369s) [noise=true , address=104.36.82.7:4912 , piece_id=NXH33OPFU7OSB2WD2MQK62P6UFELRZS4J2XUDWSBAKYDHFZUTIZA, seg_pos={0 0}, upload=16630]: Uploading piece... Jul 19 19:45:13.506354950 (20h54m19.878934305s) [upload=16630]: Next returned nil segment Jul 19 19:45:13.506372083 (20h54m19.878951438s) [upload=16630]: Waiting for error group managing segments... Jul 19 19:45:13.817836432 (20h54m20.190415837s) [noise=true , address=104.36.82.7:4912 , piece_id=NXH33OPFU7OSB2WD2MQK62P6UFELRZS4J2XUDWSBAKYDHFZUTIZA, seg_pos={0 0}, upload=16630]: Done uploading piece. err: <nil> Jul 19 19:45:13.838856270 (20h54m20.211435625s) [noise=true , address=104.36.82.13:4185 , piece_id=Y7Y2Z3QTOJVT2F6GOGETQNJRCQ5WA6ZXWC3E2QGPZNGDKXGKNYIQ, seg_pos={0 0}, upload=16630]: Done uploading piece. err: <nil> Jul 19 19:45:13.881530789 (20h54m20.254110144s) [noise=true , address=104.36.82.10:4078 , piece_id=NAXJ6FRLBHPK362HP2WS27JJEE7JM3T43HWBPW57AQEU7R5O6XGA, seg_pos={0 0}, upload=16630]: Done uploading piece. err: <nil> Jul 19 19:45:13.886929970 (20h54m20.259509385s) [noise=true , address=104.36.82.8:4971 , piece_id=TLGYY4PJHTDOLV6ZR77UZJ5MHW3J5MGJEGWJP3Z5SXRQ7NYRJBVA, seg_pos={0 0}, upload=16630]: Done uploading piece. err: <nil> Jul 19 19:45:13.904580681 (20h54m20.277160036s) [noise=true , address=104.36.82.3:4767 , piece_id=L6MAL5Z4H6ARWVEAI2W7SEWNA44R2DN6W365ZNLDSF6YXSFKOIGQ, seg_pos={0 0}, upload=16630]: Done uploading piece. err: <nil> Jul 19 19:45:13.944085776 (20h54m20.316665141s) [noise=true , address=104.36.82.16:4294 , piece_id=KVROAKA5NMPHID25WWD763DHMCB2WCVD2DXJBENLMWEIWDKWVEAA, seg_pos={0 0}, upload=16630]: Done uploading piece. err: <nil> Jul 19 19:45:13.987619149 (20h54m20.360198504s) [noise=true , address=104.36.82.9:4007 , piece_id=72VJZHEK7QC4LXBNABD2ZP5BSC6RGUCPEAH3G54HCNEZVENWDUVQ, seg_pos={0 0}, upload=16630]: Done uploading piece. err: <nil> Jul 19 19:45:14.057717069 (20h54m20.430296424s) [noise=true , address=104.36.82.11:4114 , piece_id=X3RYVF7Y74O7DK2BU5MFI6AAZH4666XNKPD23XNSXAEKJ2HSKHPQ, seg_pos={0 0}, upload=16630]: Done uploading piece. err: <nil> Jul 19 19:45:40.876578995 (20h54m47.249158351s) [noise=true , address=77.90.31.15:4699 , piece_id=3VXILCKSLCVXDD2O7EQFU5VZBS33G7TXHUKTWQ2PTWNKYHU7O5SQ, seg_pos={0 0}, upload=16630]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.876625875 (20h54m47.249205231s) [noise=true , address=77.90.31.16:4735 , piece_id=4NWJMO5SSGY4OHFD5MFPPOWDZFNKIA5YXO5TUHGYYSQ5HE4BYN4Q, seg_pos={0 0}, upload=16630]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.876668027 (20h54m47.249247402s) [noise=true , address=77.90.31.10:4517 , piece_id=C644MMDTSHF4N2EE6MFU4BBVXR63R63O62HH4K3DI6ND4ZPPB5MA, seg_pos={0 0}, upload=16630]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: write tcp 188.166.225.170:54360->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.876724104 (20h54m47.249303519s) [noise=true , address=45.91.116.14:4190 , piece_id=TK36JPUXM5YOKNJOHEFRVA7I4PYKFWN4P5IU4OO3QFECZWABNZUQ, seg_pos={0 0}, upload=16630]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: write tcp 188.166.225.170:39552->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.876809818 (20h54m47.249389173s) [noise=true , address=77.90.31.12:4589 , piece_id=JNSJMAKN3KZFCA5VYO6EBKGZK3HFR7N5MSPTDAJLXE4LA5STMXEQ, seg_pos={0 0}, upload=16630]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: rpc: dial tcp 77.90.31.12:4589: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.876858300 (20h54m47.249437655s) [noise=true , address=45.91.116.13:4154 , piece_id=ST4QCC4UTV24UXAAAXEPCFV2ISGY7NTQHNNEYDICFII3WT4NI55Q, seg_pos={0 0}, upload=16630]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.876880793 (20h54m47.249460148s) [noise=true , address=45.91.116.14:4191 , piece_id=I5KMV7RSA7FKOVMLUXKDI25SD2FIC5O6CLWWTUQMKRHHSVXRPB6A, seg_pos={0 0}, upload=16630]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.876917444 (20h54m47.249496799s) [noise=true , address=77.90.31.11:4554 , piece_id=N5U7LEC3JQZNQHU6JTHZMCXTAHBW6EEOZFOMINBGBBMF6SVXO6ZA, seg_pos={0 0}, upload=16630]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:35104->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.876950316 (20h54m47.249529671s) [seg_pos={0 0}, upload=16630]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:45:41.115144976 (20h54m47.487724321s) [upload=16630]: Done uploading segments... Jul 19 19:45:41.115167630 (20h54m47.487746985s) [upload=16630]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.239886871s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.235627484s FAIL (2025-07-19 19:45:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== [seg_pos={0 0}, upload=16626]: Begin upload segment... Jul 19 19:44:41.437426078 (20h53m47.810005433s) [seg_pos={0 0}, upload=16626]: Done begin upload segment. Jul 19 19:44:41.437445135 (20h53m47.81002449s) [noise=true , address=45.91.116.13:4172 , piece_id=VUXUFC67CVNLGXRALGYEAA2XPE44K2F74RFOLZJJIFK6H5GHWSQQ, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.437453120 (20h53m47.810032475s) [noise=true , address=77.90.31.11:4574 , piece_id=UKJULA333L5WY6KPUQ6K6ML35K6D6JRZNGV275XIZYZPBWUC47GQ, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.437494971 (20h53m47.810074316s) [noise=true , address=104.36.82.50:4030 , piece_id=XZ5VEIZAO2ICVBPFSONVTWRVQLPBMEGJQOKF7NUFSGC6MLGIIBSQ, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.437774495 (20h53m47.81035385s) [noise=true , address=45.91.95.5:4012 , piece_id=44IGCL6O6FQYVEWJ2AAYAWG7SXYCFKAO25ZT5MI552QATNC7UCFQ, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.438050083 (20h53m47.810629438s) [noise=true , address=104.36.82.5:4843 , piece_id=SR366A5XBLB4QUJ6NQ7AMPUR57ZUHRGVUX4KRQX3EOOF2DI6RC4Q, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.438577151 (20h53m47.811156496s) [noise=true , address=45.91.95.4:4015 , piece_id=H3PU42A4KSRB33Q4RUDSJXJYOENW4YEPQP4KLCOO6TCIUFH4DVRA, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.438919516 (20h53m47.811498861s) [noise=true , address=79.127.205.235:4685 , piece_id=AQ3P7LRIYNHIHS635DCQ27L2CMCLLUXSYX6OFUA45QIVKAE3YTGQ, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.439135369 (20h53m47.811714724s) [noise=true , address=104.36.82.2:4741 , piece_id=GV3VUHXFND6BWDSGYAATQV4BJKPR6K4EWPTEWX6S5PDRVO6ZCNKQ, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.439383795 (20h53m47.81196314s) [noise=true , address=79.127.205.245:4237 , piece_id=I54EYAXBAY77LI2TCMSCBA3TVPFCVMPKNA4NPE4HPPZBOL3GUZVQ, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.439582986 (20h53m47.812162341s) [noise=true , address=45.91.116.4:4947 , piece_id=HDY65MFKJYB4SFP54NCECJUVCUUHFBOSM76KN76A32QKABIT3LDA, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.439775805 (20h53m47.81235516s) [noise=true , address=45.91.116.8:4056 , piece_id=56HSHFVCI6XSFG7U2HV6GIZCLSPAFIKK7ZAFOCISH3BTIAFSN7XQ, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.439972000 (20h53m47.812551345s) [noise=true , address=45.91.95.2:4024 , piece_id=QBZTCQRMCGSESKY2FLLRUSYEOKT4JVLE7SC6T4HUAQPNFZVRVGSQ, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.440159118 (20h53m47.812738473s) [noise=true , address=45.91.95.3:4022 , piece_id=CYRC2BJVHZ5DLUR2UCDGY3TGUYGYVBCXMR6R2BDSAPVUD66C6PJA, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.440349943 (20h53m47.812929288s) [noise=true , address=77.90.31.15:4722 , piece_id=CRJVICWNBMDDNMCLYNJXOTSH5R26FXS54VFBXKMFMEHYHKTTFKWQ, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.440531139 (20h53m47.813110494s) [noise=true , address=77.90.31.11:4575 , piece_id=2AQXF3NINDXHGMGEP47IOMWCTFPHVPQ57XY34C5GRB2JHSEKFZAQ, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.440714070 (20h53m47.813293425s) [noise=true , address=104.36.82.21:4476 , piece_id=5JFXKOTSESXHQRSDDQD5YD7CP3DJU6YL6U54NXQTOQZDW4DQL6JA, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.440899956 (20h53m47.813479301s) [noise=true , address=104.36.82.41:4773 , piece_id=ITWCG6FY3UGKFXO4J4PC2ALBGISCL2X7UEA5GULH5IGUUFBNO54Q, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:41.441072135 (20h53m47.81365149s) [noise=true , address=45.91.116.9:4290 , piece_id=XLGHEKSAB34I7EAGKUN7O4BKJPS6FXYSDTGYQ5DC4ARVVMXHE6HA, seg_pos={0 0}, upload=16626]: Uploading piece... Jul 19 19:44:42.551687072 (20h53m48.924266427s) [upload=16626]: Next returned nil segment Jul 19 19:44:42.551705436 (20h53m48.924284791s) [upload=16626]: Waiting for error group managing segments... Jul 19 19:44:42.841683315 (20h53m49.21426267s) [noise=true , address=79.127.205.245:4237 , piece_id=I54EYAXBAY77LI2TCMSCBA3TVPFCVMPKNA4NPE4HPPZBOL3GUZVQ, seg_pos={0 0}, upload=16626]: Done uploading piece. err: <nil> Jul 19 19:44:42.894820864 (20h53m49.267400219s) [noise=true , address=79.127.205.235:4685 , piece_id=AQ3P7LRIYNHIHS635DCQ27L2CMCLLUXSYX6OFUA45QIVKAE3YTGQ, seg_pos={0 0}, upload=16626]: Done uploading piece. err: <nil> Jul 19 19:44:43.169271634 (20h53m49.54185099s) [noise=true , address=45.91.95.5:4012 , piece_id=44IGCL6O6FQYVEWJ2AAYAWG7SXYCFKAO25ZT5MI552QATNC7UCFQ, seg_pos={0 0}, upload=16626]: Done uploading piece. err: <nil> Jul 19 19:44:43.274222499 (20h53m49.646801854s) [noise=true , address=45.91.95.2:4024 , piece_id=QBZTCQRMCGSESKY2FLLRUSYEOKT4JVLE7SC6T4HUAQPNFZVRVGSQ, seg_pos={0 0}, upload=16626]: Done uploading piece. err: <nil> Jul 19 19:44:43.391654493 (20h53m49.764233848s) [noise=true , address=104.36.82.5:4843 , piece_id=SR366A5XBLB4QUJ6NQ7AMPUR57ZUHRGVUX4KRQX3EOOF2DI6RC4Q, seg_pos={0 0}, upload=16626]: Done uploading piece. err: <nil> Jul 19 19:44:43.441912322 (20h53m49.814491667s) [noise=true , address=104.36.82.21:4476 , piece_id=5JFXKOTSESXHQRSDDQD5YD7CP3DJU6YL6U54NXQTOQZDW4DQL6JA, seg_pos={0 0}, upload=16626]: Done uploading piece. err: <nil> Jul 19 19:44:43.468157657 (20h53m49.840737012s) [noise=true , address=104.36.82.50:4030 , piece_id=XZ5VEIZAO2ICVBPFSONVTWRVQLPBMEGJQOKF7NUFSGC6MLGIIBSQ, seg_pos={0 0}, upload=16626]: Done uploading piece. err: <nil> Jul 19 19:44:43.506606632 (20h53m49.879185987s) [noise=true , address=45.91.95.4:4015 , piece_id=H3PU42A4KSRB33Q4RUDSJXJYOENW4YEPQP4KLCOO6TCIUFH4DVRA, seg_pos={0 0}, upload=16626]: Done uploading piece. err: <nil> Jul 19 19:44:43.552431579 (20h53m49.925010944s) [noise=true , address=104.36.82.41:4773 , piece_id=ITWCG6FY3UGKFXO4J4PC2ALBGISCL2X7UEA5GULH5IGUUFBNO54Q, seg_pos={0 0}, upload=16626]: Done uploading piece. err: <nil> Jul 19 19:44:43.587472364 (20h53m49.960051719s) [noise=true , address=45.91.95.3:4022 , piece_id=CYRC2BJVHZ5DLUR2UCDGY3TGUYGYVBCXMR6R2BDSAPVUD66C6PJA, seg_pos={0 0}, upload=16626]: Done uploading piece. err: <nil> Jul 19 19:44:43.614087247 (20h53m49.986666592s) [noise=true , address=104.36.82.2:4741 , piece_id=GV3VUHXFND6BWDSGYAATQV4BJKPR6K4EWPTEWX6S5PDRVO6ZCNKQ, seg_pos={0 0}, upload=16626]: Done uploading piece. err: <nil> Jul 19 19:45:40.346013527 (20h54m46.718592882s) [noise=true , address=45.91.116.13:4172 , piece_id=VUXUFC67CVNLGXRALGYEAA2XPE44K2F74RFOLZJJIFK6H5GHWSQQ, seg_pos={0 0}, upload=16626]: Done uploading piece. err: ecclient: upload failed (node:12mGtFKf4tj7HsTRDunxBQw4WtsSqzv4wBhDExqC64qoSUSPhG5, address:45.91.116.13:4172): protocol: rpc: dial tcp 45.91.116.13:4172: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.346060056 (20h54m46.718639401s) [noise=true , address=77.90.31.15:4722 , piece_id=CRJVICWNBMDDNMCLYNJXOTSH5R26FXS54VFBXKMFMEHYHKTTFKWQ, seg_pos={0 0}, upload=16626]: Done uploading piece. err: ecclient: upload failed (node:12vx19tv5KVo9mpt896aAivK3RCEEaVnt67oRipCoU7QLa4GYzM, address:77.90.31.15:4722): protocol: rpc: dial tcp 77.90.31.15:4722: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.346088961 (20h54m46.718668316s) [noise=true , address=45.91.116.4:4947 , piece_id=HDY65MFKJYB4SFP54NCECJUVCUUHFBOSM76KN76A32QKABIT3LDA, seg_pos={0 0}, upload=16626]: Done uploading piece. err: ecclient: upload failed (node:1YTkM1w6HKKyw8z3eWomLMtJuFxsswqjZ5TmGyBeiZJE9QrFoB, address:45.91.116.4:4947): protocol: rpc: dial tcp 45.91.116.4:4947: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.346101735 (20h54m46.71868108s) [noise=true , address=45.91.116.9:4290 , piece_id=XLGHEKSAB34I7EAGKUN7O4BKJPS6FXYSDTGYQ5DC4ARVVMXHE6HA, seg_pos={0 0}, upload=16626]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: write tcp 188.166.225.170:56646->45.91.116.9:4290: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.346130641 (20h54m46.718709996s) [noise=true , address=45.91.116.8:4056 , piece_id=56HSHFVCI6XSFG7U2HV6GIZCLSPAFIKK7ZAFOCISH3BTIAFSN7XQ, seg_pos={0 0}, upload=16626]: Done uploading piece. err: ecclient: upload failed (node:12F2dgH8YUgd1uo27B9XgTVSHFirJPQPy48bd2PQY3PzdZoUeEe, address:45.91.116.8:4056): protocol: rpc: dial tcp 45.91.116.8:4056: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.346154676 (20h54m46.718734032s) [noise=true , address=77.90.31.11:4575 , piece_id=2AQXF3NINDXHGMGEP47IOMWCTFPHVPQ57XY34C5GRB2JHSEKFZAQ, seg_pos={0 0}, upload=16626]: Done uploading piece. err: ecclient: upload failed (node:1vQCT2yf3pZep8VeNBcKma579ot9EfWgaNkXQNofKjHTzdCuoc, address:77.90.31.11:4575): protocol: rpc: dial tcp 77.90.31.11:4575: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.346177090 (20h54m46.718756445s) [noise=true , address=77.90.31.11:4574 , piece_id=UKJULA333L5WY6KPUQ6K6ML35K6D6JRZNGV275XIZYZPBWUC47GQ, seg_pos={0 0}, upload=16626]: Done uploading piece. err: ecclient: upload failed (node:1pX3YNcCwaUeMstL4NNwvnKQmHbmdxLTUc8sDHwyvZGZtzmYgW, address:77.90.31.11:4574): protocol: write tcp 188.166.225.170:53012->77.90.31.11:4574: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:45:40.346217988 (20h54m46.718797343s) [seg_pos={0 0}, upload=16626]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Jul 19 19:45:40.578992238 (20h54m46.951571583s) [upload=16626]: Done uploading segments... Jul 19 19:45:40.579021734 (20h54m46.951601079s) [upload=16626]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m0.235644155s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 30.242665591s FAIL (2025-07-19 19:44:54 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== estItem_ObjectBegin Jul 19 19:44:25.006690784 (20h53m31.379270139s) [upload=16618]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:44:25.255429631 (20h53m31.628008986s) [seg_pos={0 0}, upload=16618]: Begin upload segment... Jul 19 19:44:25.255477442 (20h53m31.628056797s) [seg_pos={0 0}, upload=16618]: Done begin upload segment. Jul 19 19:44:25.255493012 (20h53m31.628072357s) [noise=true , address=45.91.116.4:4938 , piece_id=2FCSDW3XBXX2CXD5IAQ53WAIVACVHJW3UC7U4MXLXXFQHXFSSVXA, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.255499785 (20h53m31.62807914s) [noise=true , address=104.36.82.3:4767 , piece_id=SZNYT4DQWO6VAW52IGPCNKVG6C5F6AHD6DSRF5F5JOJ5HT77DBXQ, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.255569799 (20h53m31.628149154s) [noise=true , address=104.36.82.9:4007 , piece_id=JQYQE5TOQSBJTQ4JJWL34AGFQFD6MSUZMGT44F4YTFUUXWTHPNUQ, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.255848061 (20h53m31.628427416s) [noise=true , address=77.90.31.15:4699 , piece_id=GFKIEF4O562HTYFNPQTKIH4M5VISHH4E5LK3RBTRALIZS6M5FVJQ, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.256095174 (20h53m31.628674529s) [noise=true , address=104.36.82.10:4077 , piece_id=Y6LOWQVK3VV5UUCB227QLFDXMSIBEPK6QN5C7GY2RJ7SBOYCZMUQ, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.256354139 (20h53m31.628933494s) [noise=true , address=45.91.116.12:4225 , piece_id=IJ6IHH76J7WDAKTUCA6IRJWRYDP377Q7526TX4NM4R5REN2G3DMA, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.256638384 (20h53m31.629217759s) [noise=true , address=77.90.31.16:4734 , piece_id=GQC4EQKUKSTTIKAZCNDZQJKFC7TR7TZMFCPE4MMGPRBZQ5V72WCQ, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.257025644 (20h53m31.629605s) [noise=true , address=77.90.31.11:4553 , piece_id=YEVQY7JG2GOCCBSDLDCQVY3VDVEN3DKXCIA32OFOBPKLUHC6DUJA, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.257085098 (20h53m31.629664463s) [noise=true , address=77.90.31.21:4877 , piece_id=3QR3YGGMMXI4FB5QJGF3IEBYLTOADIK6ALO443FTEHAUL5ISB6BA, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.257258680 (20h53m31.629838035s) [noise=true , address=104.36.82.6:4873 , piece_id=ONPWODGA2XWEYE56PNVQO7GELHFTFYQSALT2VD32SZ2QUHGT7SUQ, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.257360154 (20h53m31.629939509s) [noise=true , address=77.90.31.14:4662 , piece_id=Z35OUTC6PHQH2TWRUKMWK7GTS5ST73BKXZVKFGMBEPYZOKKQV5GA, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.257450607 (20h53m31.630029962s) [noise=true , address=77.90.31.10:4519 , piece_id=MQWXE7G5UJSWQI4IBCC32VV5QY4GOXCGOTD4VNULVCFDH57TMAOA, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.257635952 (20h53m31.630215307s) [noise=true , address=45.91.116.7:4009 , piece_id=LPQN54YPYD5ME34THF3AOB2N3B3TEFUWKKDMMBWTV5KQH6OVZ3TQ, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.257843168 (20h53m31.630422523s) [noise=true , address=104.36.82.8:4970 , piece_id=6KEMXAUDZAOI6KECVMOU2TNYLCFPGG4656LCQSAGAVW5PL65FWRQ, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.258036207 (20h53m31.630615552s) [noise=true , address=104.36.82.12:4151 , piece_id=ADD5NJMDVSFS7V2GUOVWMBJBXBQH7HRXJRZ6Z4Y3TBKFVCY2TBRQ, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:25.258225610 (20h53m31.630804965s) [noise=true , address=45.91.116.4:4939 , piece_id=JDQJWL2PTUYZRTNCHIBR7PZXIYI5UZ75CQEFIIOM2VECFN2AD34A, seg_pos={0 0}, upload=16618]: Uploading piece... Jul 19 19:44:27.007579984 (20h53m33.380159329s) [upload=16618]: Next returned nil segment Jul 19 19:44:27.007605693 (20h53m33.380185048s) [upload=16618]: Waiting for error group managing segments... Jul 19 19:44:27.302039998 (20h53m33.674619343s) [noise=true , address=104.36.82.9:4007 , piece_id=JQYQE5TOQSBJTQ4JJWL34AGFQFD6MSUZMGT44F4YTFUUXWTHPNUQ, seg_pos={0 0}, upload=16618]: Done uploading piece. err: <nil> Jul 19 19:44:27.335706232 (20h53m33.708285587s) [noise=true , address=104.36.82.10:4077 , piece_id=Y6LOWQVK3VV5UUCB227QLFDXMSIBEPK6QN5C7GY2RJ7SBOYCZMUQ, seg_pos={0 0}, upload=16618]: Done uploading piece. err: <nil> Jul 19 19:44:27.372599832 (20h53m33.745179187s) [noise=true , address=104.36.82.8:4970 , piece_id=6KEMXAUDZAOI6KECVMOU2TNYLCFPGG4656LCQSAGAVW5PL65FWRQ, seg_pos={0 0}, upload=16618]: Done uploading piece. err: <nil> Jul 19 19:44:27.375796999 (20h53m33.748376354s) [noise=true , address=104.36.82.6:4873 , piece_id=ONPWODGA2XWEYE56PNVQO7GELHFTFYQSALT2VD32SZ2QUHGT7SUQ, seg_pos={0 0}, upload=16618]: Done uploading piece. err: <nil> Jul 19 19:44:27.396845722 (20h53m33.769425077s) [noise=true , address=104.36.82.12:4151 , piece_id=ADD5NJMDVSFS7V2GUOVWMBJBXBQH7HRXJRZ6Z4Y3TBKFVCY2TBRQ, seg_pos={0 0}, upload=16618]: Done uploading piece. err: <nil> Jul 19 19:44:27.498951852 (20h53m33.871531207s) [noise=true , address=104.36.82.3:4767 , piece_id=SZNYT4DQWO6VAW52IGPCNKVG6C5F6AHD6DSRF5F5JOJ5HT77DBXQ, seg_pos={0 0}, upload=16618]: Done uploading piece. err: <nil> Jul 19 19:44:54.354079629 (20h54m0.726658984s) [noise=true , address=77.90.31.21:4877 , piece_id=3QR3YGGMMXI4FB5QJGF3IEBYLTOADIK6ALO443FTEHAUL5ISB6BA, seg_pos={0 0}, upload=16618]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:54.354126738 (20h54m0.726706093s) [noise=true , address=77.90.31.14:4662 , piece_id=Z35OUTC6PHQH2TWRUKMWK7GTS5ST73BKXZVKFGMBEPYZOKKQV5GA, seg_pos={0 0}, upload=16618]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:54.354169420 (20h54m0.726748775s) [noise=true , address=77.90.31.16:4734 , piece_id=GQC4EQKUKSTTIKAZCNDZQJKFC7TR7TZMFCPE4MMGPRBZQ5V72WCQ, seg_pos={0 0}, upload=16618]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:54.354263440 (20h54m0.726842795s) [noise=true , address=77.90.31.10:4519 , piece_id=MQWXE7G5UJSWQI4IBCC32VV5QY4GOXCGOTD4VNULVCFDH57TMAOA, seg_pos={0 0}, upload=16618]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: rpc: dial tcp 77.90.31.10:4519: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:54.354307815 (20h54m0.72688717s) [noise=true , address=45.91.116.4:4939 , piece_id=JDQJWL2PTUYZRTNCHIBR7PZXIYI5UZ75CQEFIIOM2VECFN2AD34A, seg_pos={0 0}, upload=16618]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:34726->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:54.354407276 (20h54m0.726986631s) [noise=true , address=45.91.116.12:4225 , piece_id=IJ6IHH76J7WDAKTUCA6IRJWRYDP377Q7526TX4NM4R5REN2G3DMA, seg_pos={0 0}, upload=16618]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:54.354441341 (20h54m0.727020696s) [noise=true , address=77.90.31.11:4553 , piece_id=YEVQY7JG2GOCCBSDLDCQVY3VDVEN3DKXCIA32OFOBPKLUHC6DUJA, seg_pos={0 0}, upload=16618]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: rpc: dial tcp 77.90.31.11:4553: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:54.354499452 (20h54m0.727078797s) [noise=true , address=45.91.116.4:4938 , piece_id=2FCSDW3XBXX2CXD5IAQ53WAIVACVHJW3UC7U4MXLXXFQHXFSSVXA, seg_pos={0 0}, upload=16618]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Jul 19 19:44:54.354503429 (20h54m0.727082784s) [noise=true , address=45.91.116.7:4009 , piece_id=LPQN54YPYD5ME34THF3AOB2N3B3TEFUWKKDMMBWTV5KQH6OVZ3TQ, seg_pos={0 0}, upload=16618]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:54.354530301 (20h54m0.727109656s) [noise=true , address=77.90.31.15:4699 , piece_id=GFKIEF4O562HTYFNPQTKIH4M5VISHH4E5LK3RBTRALIZS6M5FVJQ, seg_pos={0 0}, upload=16618]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 188.166.225.170:50706->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:54.354549137 (20h54m0.727128492s) [seg_pos={0 0}, upload=16618]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 19:44:54.594542749 (20h54m0.967122104s) [upload=16618]: Done uploading segments... Jul 19 19:44:54.594565363 (20h54m0.967144718s) [upload=16618]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration:* 30.242677423s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.260466204s FAIL (2025-07-19 19:44:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== seg_pos={0 0}, upload=16612]: Begin upload segment... Jul 19 19:43:40.960387934 (20h52m47.332967289s) [seg_pos={0 0}, upload=16612]: Done begin upload segment. Jul 19 19:43:40.960426809 (20h52m47.333006164s) [noise=true , address=64.49.13.27:4966 , piece_id=MS52SIOYQMT5BQPR473EA4W5VXPPZM5QRQRIPFD46VV2JQNRGGMA, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.960588058 (20h52m47.333167413s) [noise=true , address=104.36.82.56:4253 , piece_id=GFLNUHJOGX4OPRUGBLSZ7G5SMC6PIJQP6IZK45YTOIVFZLXBKSWQ, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.961169740 (20h52m47.333749095s) [noise=true , address=104.36.82.9:4035 , piece_id=LZ5Q6LD5OZ6ETZ4NR3OB6HX35VRYL26FECNERMSATUCMOQPNGVNQ, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.961376285 (20h52m47.33395564s) [noise=true , address=77.90.31.21:4899 , piece_id=4SRYMK2YCY2NRX2GTPGNCKAQRYD6YSCWI6DILUFKCDXI4KM5IMAA, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.961740312 (20h52m47.334319667s) [noise=true , address=64.49.13.138:4482 , piece_id=RYMHPMSRHQBVIF6EPM775Y7V5XSCXB7LKPCWZ7Z7YCMB7FGP4PJA, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.962008625 (20h52m47.33458798s) [noise=true , address=45.91.116.8:4051 , piece_id=HMQRTVVPACCO5EMWNAG6RTB723FHOIMAFTC6FCX4ZFWMMJTWTAJA, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.962268332 (20h52m47.334847677s) [noise=true , address=79.127.205.245:4217 , piece_id=H6SWXUGKSFVG3DFJIMZD2QA75KPHAWWPFW7ILSKNHDGZROAD5BDA, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.962407378 (20h52m47.334986733s) [noise=true , address=45.91.95.4:4019 , piece_id=PBFMRZLWBCT6A4CC3ZINKA2XZP6SB3XGN6UPUKWHI5AXO62MAHTA, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.962475558 (20h52m47.335054913s) [noise=true , address=45.91.116.7:4035 , piece_id=VUT3GBH6HAETMXCRG56ZUHY4TG5AYRCLEHV4FBCJ5W6MLQ2PG3NA, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.962503602 (20h52m47.335082957s) [noise=true , address=45.91.116.14:4219 , piece_id=WHT4IXS4VEN42YYFZB3VUPRUE2EZ4BN5DXI7PV6XHFZ5Q436T2DQ, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.962780622 (20h52m47.335359977s) [noise=true , address=104.36.82.40:4973 , piece_id=A7CSSP5IJR4KFOWYIAGT7ZZHPWJCYTFL2Q7OC5EA7L5NRYTIB6OQ, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.962940207 (20h52m47.335519553s) [noise=true , address=77.90.31.10:4547 , piece_id=JGRAV3FUXCYVFOOIMTVG635RHYY4235IKSDUYUUIBIWL4X25A5WA, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.963011995 (20h52m47.33559135s) [noise=true , address=77.90.31.12:4600 , piece_id=4OEUSOWEVKDRZHMUYEK5AX72PMAMWGCB7HFBLT47WM2YPIQGBKRA, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.963291360 (20h52m47.335870715s) [noise=true , address=45.91.116.5:4321 , piece_id=PRIQOOOQG4Y6VSVZWSM3EUK2N5XCANWJSQSSMROJNZKE4XRVUHMA, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.963612554 (20h52m47.336191899s) [noise=true , address=45.91.95.5:4035 , piece_id=U6TKLWJMQMLCYARW4QSI7WSBHNNZYVX4IXZHN2OKMKP2XMUPH6JQ, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.963918049 (20h52m47.336497404s) [noise=true , address=79.127.201.214:4308 , piece_id=HUFA53EEH536JZKLZ3PZXIIC6GMZFUW4GSYE2Z5OZP7X2PQ6EIKQ, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.964193195 (20h52m47.33677255s) [noise=true , address=79.127.201.215:4344 , piece_id=GBMIMRFAXMYAD4AIPZWMYEUO3W3N6F2LDKPGHDIZBGL57DMMJI5A, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:40.964498529 (20h52m47.337077884s) [noise=true , address=79.127.220.98:4432 , piece_id=AVZ55J66AHBJ5IVRM6IZ65ZFUIHMRWZPDI3BAWT3JQAUBUR3HRVA, seg_pos={0 0}, upload=16612]: Uploading piece... Jul 19 19:43:42.068279471 (20h52m48.440858816s) [upload=16612]: Next returned nil segment Jul 19 19:43:42.068292556 (20h52m48.440871911s) [upload=16612]: Waiting for error group managing segments... Jul 19 19:43:42.359221856 (20h52m48.731801211s) [noise=true , address=79.127.205.245:4217 , piece_id=H6SWXUGKSFVG3DFJIMZD2QA75KPHAWWPFW7ILSKNHDGZROAD5BDA, seg_pos={0 0}, upload=16612]: Done uploading piece. err: <nil> Jul 19 19:43:42.415089558 (20h52m48.787668913s) [noise=true , address=64.49.13.138:4482 , piece_id=RYMHPMSRHQBVIF6EPM775Y7V5XSCXB7LKPCWZ7Z7YCMB7FGP4PJA, seg_pos={0 0}, upload=16612]: Done uploading piece. err: <nil> Jul 19 19:43:42.457482389 (20h52m48.830061745s) [noise=true , address=64.49.13.27:4966 , piece_id=MS52SIOYQMT5BQPR473EA4W5VXPPZM5QRQRIPFD46VV2JQNRGGMA, seg_pos={0 0}, upload=16612]: Done uploading piece. err: <nil> Jul 19 19:43:42.669692294 (20h52m49.042271649s) [noise=true , address=45.91.95.5:4035 , piece_id=U6TKLWJMQMLCYARW4QSI7WSBHNNZYVX4IXZHN2OKMKP2XMUPH6JQ, seg_pos={0 0}, upload=16612]: Done uploading piece. err: <nil> Jul 19 19:43:42.730338939 (20h52m49.102918295s) [noise=true , address=45.91.95.4:4019 , piece_id=PBFMRZLWBCT6A4CC3ZINKA2XZP6SB3XGN6UPUKWHI5AXO62MAHTA, seg_pos={0 0}, upload=16612]: Done uploading piece. err: <nil> Jul 19 19:43:42.739717895 (20h52m49.11229725s) [noise=true , address=79.127.220.98:4432 , piece_id=AVZ55J66AHBJ5IVRM6IZ65ZFUIHMRWZPDI3BAWT3JQAUBUR3HRVA, seg_pos={0 0}, upload=16612]: Done uploading piece. err: <nil> Jul 19 19:43:42.856044474 (20h52m49.228623829s) [noise=true , address=79.127.201.215:4344 , piece_id=GBMIMRFAXMYAD4AIPZWMYEUO3W3N6F2LDKPGHDIZBGL57DMMJI5A, seg_pos={0 0}, upload=16612]: Done uploading piece. err: <nil> Jul 19 19:43:42.864784146 (20h52m49.237363501s) [noise=true , address=79.127.201.214:4308 , piece_id=HUFA53EEH536JZKLZ3PZXIIC6GMZFUW4GSYE2Z5OZP7X2PQ6EIKQ, seg_pos={0 0}, upload=16612]: Done uploading piece. err: <nil> Jul 19 19:43:42.894306050 (20h52m49.266885406s) [noise=true , address=104.36.82.40:4973 , piece_id=A7CSSP5IJR4KFOWYIAGT7ZZHPWJCYTFL2Q7OC5EA7L5NRYTIB6OQ, seg_pos={0 0}, upload=16612]: Done uploading piece. err: <nil> Jul 19 19:43:42.983843925 (20h52m49.35642328s) [noise=true , address=104.36.82.9:4035 , piece_id=LZ5Q6LD5OZ6ETZ4NR3OB6HX35VRYL26FECNERMSATUCMOQPNGVNQ, seg_pos={0 0}, upload=16612]: Done uploading piece. err: <nil> Jul 19 19:43:43.028347404 (20h52m49.400926749s) [noise=true , address=104.36.82.56:4253 , piece_id=GFLNUHJOGX4OPRUGBLSZ7G5SMC6PIJQP6IZK45YTOIVFZLXBKSWQ, seg_pos={0 0}, upload=16612]: Done uploading piece. err: <nil> Jul 19 19:44:39.858959355 (20h53m46.23153871s) [noise=true , address=77.90.31.10:4547 , piece_id=JGRAV3FUXCYVFOOIMTVG635RHYY4235IKSDUYUUIBIWL4X25A5WA, seg_pos={0 0}, upload=16612]: Done uploading piece. err: ecclient: upload failed (node:12hkdk6W2VmYLGra1GpBVJjkypiydor2f1LDjjGL2Kkb6fJqENZ, address:77.90.31.10:4547): protocol: rpc: dial tcp 77.90.31.10:4547: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:39.859016895 (20h53m46.23159625s) [noise=true , address=45.91.116.14:4219 , piece_id=WHT4IXS4VEN42YYFZB3VUPRUE2EZ4BN5DXI7PV6XHFZ5Q436T2DQ, seg_pos={0 0}, upload=16612]: Done uploading piece. err: ecclient: upload failed (node:1gdoFPze5qLmQfouMyko8R8UNhFcNDMgawCWWySyQ8kdrfUXPr, address:45.91.116.14:4219): protocol: rpc: dial tcp 45.91.116.14:4219: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:39.859065889 (20h53m46.231645244s) [noise=true , address=77.90.31.12:4600 , piece_id=4OEUSOWEVKDRZHMUYEK5AX72PMAMWGCB7HFBLT47WM2YPIQGBKRA, seg_pos={0 0}, upload=16612]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: write tcp 188.166.225.170:54200->77.90.31.12:4600: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:39.859113840 (20h53m46.231693205s) [noise=true , address=45.91.116.8:4051 , piece_id=HMQRTVVPACCO5EMWNAG6RTB723FHOIMAFTC6FCX4ZFWMMJTWTAJA, seg_pos={0 0}, upload=16612]: Done uploading piece. err: ecclient: upload failed (node:1preT33ei4yVXXYVLHhaUbHbiyL14oHFEiof6cCWPig83W6CT3, address:45.91.116.8:4051): protocol: rpc: dial tcp 45.91.116.8:4051: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:39.859150300 (20h53m46.231729645s) [noise=true , address=45.91.116.7:4035 , piece_id=VUT3GBH6HAETMXCRG56ZUHY4TG5AYRCLEHV4FBCJ5W6MLQ2PG3NA, seg_pos={0 0}, upload=16612]: Done uploading piece. err: ecclient: upload failed (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg, address:45.91.116.7:4035): protocol: rpc: dial tcp 45.91.116.7:4035: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:39.859174607 (20h53m46.231753962s) [noise=true , address=45.91.116.5:4321 , piece_id=PRIQOOOQG4Y6VSVZWSM3EUK2N5XCANWJSQSSMROJNZKE4XRVUHMA, seg_pos={0 0}, upload=16612]: Done uploading piece. err: ecclient: upload failed (node:12m3Lw8yL9wVq38nW514iaLooMVgikbQBGA9vrFsr66jx5w1xXj, address:45.91.116.5:4321): protocol: rpc: dial tcp 45.91.116.5:4321: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:39.859206347 (20h53m46.231785702s) [noise=true , address=77.90.31.21:4899 , piece_id=4SRYMK2YCY2NRX2GTPGNCKAQRYD6YSCWI6DILUFKCDXI4KM5IMAA, seg_pos={0 0}, upload=16612]: Done uploading piece. err: ecclient: upload failed (node:12UjFkFn2UEvr57LDorcijDCvkBkkKitgZEFx6Yqfk5gMStQvJG, address:77.90.31.21:4899): protocol: write tcp 188.166.225.170:49806->77.90.31.21:4899: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:39.859237296 (20h53m46.231816652s) [seg_pos={0 0}, upload=16612]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Jul 19 19:44:40.116972865 (20h53m46.48955221s) [upload=16612]: Done uploading segments... Jul 19 19:44:40.117004656 (20h53m46.489584011s) [upload=16612]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m0.260483327s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.252422019s FAIL (2025-07-19 19:44:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== oad=16613]: Begin upload segment... Jul 19 19:43:54.747580599 (20h53m1.120159954s) [seg_pos={0 0}, upload=16613]: Done begin upload segment. Jul 19 19:43:54.747601248 (20h53m1.120180603s) [noise=true , address=77.90.31.14:4663 , piece_id=75RK2AG2VWXDUSPYPGWNZKJMR44ECCPH45RO7ZHECV6LJ532MTLA, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.747665221 (20h53m1.120244576s) [noise=true , address=77.90.31.18:4806 , piece_id=IIX64XALNNMFKOM7UDNY3UN4HJ5QBBMVCMSFG25FSOHEVXLL5HJQ, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.747952170 (20h53m1.120531525s) [noise=true , address=45.91.116.13:4154 , piece_id=JVGEMFTYEQWAV6H7RGHZZU65MIUTD64BTHFYOHT5UGOAITV73BZQ, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.748220013 (20h53m1.120799368s) [noise=true , address=77.90.31.16:4734 , piece_id=AHYTPHVSHGNGJGBBID67LSM7MY5UF66WQHTCSXCV6VWE2X2Q7D4A, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.748231735 (20h53m1.12081108s) [noise=true , address=104.36.82.8:4970 , piece_id=6TKRB43JRDKKZD4UZPREP3IIZBMIR5S2GYAQHEHWSGS6O4IBJVGQ, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.748485189 (20h53m1.121064544s) [noise=true , address=77.90.31.21:4879 , piece_id=MWJ5KXGYVGYV26RRC5GN5VTLLPL2K2SIX3UVTVHBTMONTP5UEJLA, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.748540024 (20h53m1.121119369s) [noise=true , address=104.36.82.14:4222 , piece_id=JG4NLQZTL4TYDGNFLEF4MOSGQNJWPI5LTIOHN6VVHEZ44IZMGHRA, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.748729166 (20h53m1.121308521s) [noise=true , address=45.91.116.14:4190 , piece_id=MJ2EWYZ2YJFQ5HRPBZRHCRBP6TNOENEZLMA44WEW4U2AYAHGV2IA, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.748796375 (20h53m1.12137573s) [noise=true , address=104.36.82.7:4909 , piece_id=UAFYXU7M46R4Z6V5QBLHB5X4RWESQ255SIG5RBD7NLGPA5SHFTNA, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.748958856 (20h53m1.121538211s) [noise=true , address=104.36.82.6:4875 , piece_id=AJC2R54WUH3AVBJH3NJHBW4GE736D7ZWR3HTHBTDR3NIT5HUUWIA, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.749135053 (20h53m1.121714408s) [noise=true , address=45.91.116.4:4937 , piece_id=OE7WSQJ2HWPT6GZJ3ECHDT3BH4ZMBTRKZQRMYEYWU4K7IVEZ55IQ, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.749191581 (20h53m1.121770936s) [noise=true , address=77.90.31.13:4627 , piece_id=RQ6NLIHUFV2HWJIJMVVFPKEYY3OBTIEX46A4SOFKLB7M37KMW4SA, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.749392736 (20h53m1.121972081s) [noise=true , address=45.91.116.10:4083 , piece_id=5HVTLHK54ZW56LTVMA2BM6B6Q4RUQK4FIG7OQCRBPV7O7UPJMZ3Q, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.749422794 (20h53m1.122002149s) [noise=true , address=104.36.82.9:4006 , piece_id=QIS4J6EZEWSGPLO7MH5A643OMR2MXNHJJ6ZZQJ2ORBNV7R426BIQ, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.749664095 (20h53m1.12224344s) [noise=true , address=45.91.116.9:4261 , piece_id=THBCG7VZK3AXZQDPSYE2SJDHSBB25KSO4INJIEFDNCZQFXVYA3TA, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:54.750180904 (20h53m1.122760249s) [noise=true , address=45.91.116.7:4009 , piece_id=DAN6M472WLZNOLEP7EBMA24HQ3CXM5TZQDWN6KRHCN7WD6T4ATOQ, seg_pos={0 0}, upload=16613]: Uploading piece... Jul 19 19:43:56.517322591 (20h53m2.889901946s) [upload=16613]: Next returned nil segment Jul 19 19:43:56.517350614 (20h53m2.889929969s) [upload=16613]: Waiting for error group managing segments... Jul 19 19:43:56.807459304 (20h53m3.180038659s) [noise=true , address=104.36.82.14:4222 , piece_id=JG4NLQZTL4TYDGNFLEF4MOSGQNJWPI5LTIOHN6VVHEZ44IZMGHRA, seg_pos={0 0}, upload=16613]: Done uploading piece. err: <nil> Jul 19 19:43:56.855761631 (20h53m3.228340996s) [noise=true , address=104.36.82.8:4970 , piece_id=6TKRB43JRDKKZD4UZPREP3IIZBMIR5S2GYAQHEHWSGS6O4IBJVGQ, seg_pos={0 0}, upload=16613]: Done uploading piece. err: <nil> Jul 19 19:43:56.874369474 (20h53m3.246948829s) [noise=true , address=104.36.82.7:4909 , piece_id=UAFYXU7M46R4Z6V5QBLHB5X4RWESQ255SIG5RBD7NLGPA5SHFTNA, seg_pos={0 0}, upload=16613]: Done uploading piece. err: <nil> Jul 19 19:43:56.882828749 (20h53m3.255408105s) [noise=true , address=104.36.82.6:4875 , piece_id=AJC2R54WUH3AVBJH3NJHBW4GE736D7ZWR3HTHBTDR3NIT5HUUWIA, seg_pos={0 0}, upload=16613]: Done uploading piece. err: <nil> Jul 19 19:43:56.887316617 (20h53m3.259895962s) [noise=true , address=104.36.82.9:4006 , piece_id=QIS4J6EZEWSGPLO7MH5A643OMR2MXNHJJ6ZZQJ2ORBNV7R426BIQ, seg_pos={0 0}, upload=16613]: Done uploading piece. err: <nil> Jul 19 19:44:23.872642142 (20h53m30.245221547s) [noise=true , address=45.91.116.13:4154 , piece_id=JVGEMFTYEQWAV6H7RGHZZU65MIUTD64BTHFYOHT5UGOAITV73BZQ, seg_pos={0 0}, upload=16613]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: rpc: dial tcp 45.91.116.13:4154: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:23.872715943 (20h53m30.245295389s) [noise=true , address=45.91.116.14:4190 , piece_id=MJ2EWYZ2YJFQ5HRPBZRHCRBP6TNOENEZLMA44WEW4U2AYAHGV2IA, seg_pos={0 0}, upload=16613]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: rpc: dial tcp 45.91.116.14:4190: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:23.872743276 (20h53m30.245322691s) [noise=true , address=77.90.31.13:4627 , piece_id=RQ6NLIHUFV2HWJIJMVVFPKEYY3OBTIEX46A4SOFKLB7M37KMW4SA, seg_pos={0 0}, upload=16613]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: rpc: dial tcp 77.90.31.13:4627: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:23.872813430 (20h53m30.245392835s) [noise=true , address=45.91.116.7:4009 , piece_id=DAN6M472WLZNOLEP7EBMA24HQ3CXM5TZQDWN6KRHCN7WD6T4ATOQ, seg_pos={0 0}, upload=16613]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: write tcp 188.166.225.170:46190->45.91.116.7:4009: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:23.872826064 (20h53m30.245405469s) [noise=true , address=45.91.116.10:4083 , piece_id=5HVTLHK54ZW56LTVMA2BM6B6Q4RUQK4FIG7OQCRBPV7O7UPJMZ3Q, seg_pos={0 0}, upload=16613]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: rpc: dial tcp 45.91.116.10:4083: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:23.872902480 (20h53m30.245481896s) [noise=true , address=77.90.31.18:4806 , piece_id=IIX64XALNNMFKOM7UDNY3UN4HJ5QBBMVCMSFG25FSOHEVXLL5HJQ, seg_pos={0 0}, upload=16613]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: rpc: dial tcp 77.90.31.18:4806: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:23.872981031 (20h53m30.245560416s) [noise=true , address=77.90.31.21:4879 , piece_id=MWJ5KXGYVGYV26RRC5GN5VTLLPL2K2SIX3UVTVHBTMONTP5UEJLA, seg_pos={0 0}, upload=16613]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:23.872991781 (20h53m30.245571196s) [noise=true , address=77.90.31.16:4734 , piece_id=AHYTPHVSHGNGJGBBID67LSM7MY5UF66WQHTCSXCV6VWE2X2Q7D4A, seg_pos={0 0}, upload=16613]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:23.873000848 (20h53m30.245580213s) [noise=true , address=45.91.116.4:4937 , piece_id=OE7WSQJ2HWPT6GZJ3ECHDT3BH4ZMBTRKZQRMYEYWU4K7IVEZ55IQ, seg_pos={0 0}, upload=16613]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: rpc: dial tcp 45.91.116.4:4937: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:23.873035074 (20h53m30.245614489s) [noise=true , address=45.91.116.9:4261 , piece_id=THBCG7VZK3AXZQDPSYE2SJDHSBB25KSO4INJIEFDNCZQFXVYA3TA, seg_pos={0 0}, upload=16613]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: rpc: dial tcp 45.91.116.9:4261: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:23.873075722 (20h53m30.245655127s) [noise=true , address=77.90.31.14:4663 , piece_id=75RK2AG2VWXDUSPYPGWNZKJMR44ECCPH45RO7ZHECV6LJ532MTLA, seg_pos={0 0}, upload=16613]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: write tcp 188.166.225.170:39672->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:44:23.873115789 (20h53m30.245695194s) [seg_pos={0 0}, upload=16613]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Jul 19 19:44:24.123733608 (20h53m30.496313003s) [upload=16613]: Done uploading segments... Jul 19 19:44:24.123765370 (20h53m30.496344755s) [upload=16613]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.252439171s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.241593598s FAIL (2025-07-19 19:43:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 1.19:4841: use of closed network connection; write tcp 188.166.225.170:44606->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44606->77.90.31.19:4841: use of closed network connection Jul 19 19:42:57.648231301 (20h52m4.020810656s) [seg_pos={0 0}, upload=16594]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 19:42:57.895222032 (20h52m4.267801397s) [upload=16594]: Done uploading segments... Jul 19 19:42:57.895369914 (20h52m4.267949279s) [upload=16594]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Jul 19 19:43:06.182654250 (20h52m12.555233595s) [upload=16602]: Starting upload Jul 19 19:43:06.182711039 (20h52m12.555290394s) [upload=16602]: Done starting upload Jul 19 19:43:06.182801081 (20h52m12.555380436s) [upload=16602]: Uploading segments... Jul 19 19:43:06.182896985 (20h52m12.55547634s) [upload=16602]: Got next segment. Inline: false Jul 19 19:43:06.182935078 (20h52m12.555514433s) [upload=16602]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:43:06.182939536 (20h52m12.555518891s) [upload=16602]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:43:06.429339055 (20h52m12.80191841s) [seg_pos={0 0}, upload=16602]: Begin upload segment... Jul 19 19:43:06.429376367 (20h52m12.801955722s) [seg_pos={0 0}, upload=16602]: Done begin upload segment. Jul 19 19:43:06.429401114 (20h52m12.801980469s) [noise=true , address=45.91.116.12:4226 , piece_id=C46JETQMZFEOQWHGYTOIZHCWPDB2N22CYFLK7AQWIZY5RKQOYXAA, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.429454656 (20h52m12.802034001s) [noise=true , address=45.91.116.9:4261 , piece_id=ORGY5K4DHN3VGKBQKQCWNEF7QC4TSITV2YZUS2Y6LRVRMKXPWAJA, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.429747156 (20h52m12.802326511s) [noise=true , address=77.90.31.18:4805 , piece_id=BO3EOYNIHA3QR6RWTWDYDE352ORRUXOWXTH4RUN7RP7XMWG7HFCQ, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.429901201 (20h52m12.802480556s) [noise=true , address=104.36.82.2:4730 , piece_id=7QDWKGLJJ6J4F2JZSZ22DBDIZEEH3RBR6RX5IWZ2V25JUL2RGWYA, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.430044204 (20h52m12.802623559s) [noise=true , address=104.36.82.5:4839 , piece_id=5GC664KPRMT7UWO6ZMWLCK66R2BS3VV6YH5TL6UTSHHCYMDKX7RA, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.430185014 (20h52m12.802764359s) [noise=true , address=77.90.31.17:4770 , piece_id=YK3LO2LE5PS25B5U3MJHAAGHJQQR3CBNO2QYBEYZVZ4AXYM6NYQQ, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.430323438 (20h52m12.802902794s) [noise=true , address=77.90.31.10:4517 , piece_id=ECRVBANP3AQAWZBV3FEWF6FXWNHHZNGGOJXARJHFOKQMMPPM6V7Q, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.430450211 (20h52m12.803029566s) [noise=true , address=104.36.82.6:4873 , piece_id=DRT5EUOQPY24KRCRU7N3EFGB7KSVAKGLIKXFRNIX6JABX7EXLGZQ, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.430603745 (20h52m12.8031831s) [noise=true , address=104.36.82.15:4259 , piece_id=JB2TOANX532U76QTN4OVYXQYUXUWZWMME2BS6XPWMITOTADLRLVQ, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.430708024 (20h52m12.803287379s) [noise=true , address=77.90.31.12:4589 , piece_id=JI6CQS7ZO3BB7A24QIQJAIRQ3O6MHJJSLK5VKGCIVTTIP3FGLGUA, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.430864274 (20h52m12.803443629s) [noise=true , address=104.36.82.3:4767 , piece_id=LJOY4D4E626ZKXO5DBNEFWZYJA4SNWB6SQSUYR6E6ZCBCLYDALAA, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.430925290 (20h52m12.803504645s) [noise=true , address=104.36.82.7:4909 , piece_id=FQ72QZ4Z2RATGSZT4ZF35ZXCDVINOQVYGKGJTCMD6CARBE2WJXKQ, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.431100796 (20h52m12.803680141s) [noise=true , address=45.91.116.13:4154 , piece_id=HU44VLLUBD57S2VJJ6KSBOEQMLAQWC73OXTBUZ2YIWMVSSGZWBAA, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.431138919 (20h52m12.803718274s) [noise=true , address=77.90.31.11:4553 , piece_id=BCIRUINTOO4PU3HUXZWNWMUAPF4XJHJGM5JTUKRNSUK6A23Q55BQ, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.431381503 (20h52m12.803960858s) [noise=true , address=104.36.82.16:4295 , piece_id=UMGCMTDS7NLLEVRGPCOZ7OC2W7XG3PAVV4DZT33X2KEEUGLKRURQ, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:06.431611443 (20h52m12.804190798s) [noise=true , address=104.36.82.5:4838 , piece_id=6K6DSZET34IU6GLSQU4OGYI6T754TQKMBUCRWELX6FX7YIZ2DYYQ, seg_pos={0 0}, upload=16602]: Uploading piece... Jul 19 19:43:08.131433687 (20h52m14.504013042s) [upload=16602]: Next returned nil segment Jul 19 19:43:08.131452652 (20h52m14.504032007s) [upload=16602]: Waiting for error group managing segments... Jul 19 19:43:08.486276953 (20h52m14.858856308s) [noise=true , address=45.91.116.12:4226 , piece_id=C46JETQMZFEOQWHGYTOIZHCWPDB2N22CYFLK7AQWIZY5RKQOYXAA, seg_pos={0 0}, upload=16602]: Done uploading piece. err: <nil> Jul 19 19:43:08.492751421 (20h52m14.865330776s) [noise=true , address=104.36.82.6:4873 , piece_id=DRT5EUOQPY24KRCRU7N3EFGB7KSVAKGLIKXFRNIX6JABX7EXLGZQ, seg_pos={0 0}, upload=16602]: Done uploading piece. err: <nil> Jul 19 19:43:08.501144620 (20h52m14.873723975s) [noise=true , address=104.36.82.2:4730 , piece_id=7QDWKGLJJ6J4F2JZSZ22DBDIZEEH3RBR6RX5IWZ2V25JUL2RGWYA, seg_pos={0 0}, upload=16602]: Done uploading piece. err: <nil> Jul 19 19:43:08.569996785 (20h52m14.94257614s) [noise=true , address=104.36.82.15:4259 , piece_id=JB2TOANX532U76QTN4OVYXQYUXUWZWMME2BS6XPWMITOTADLRLVQ, seg_pos={0 0}, upload=16602]: Done uploading piece. err: <nil> Jul 19 19:43:08.571850462 (20h52m14.944429818s) [noise=true , address=104.36.82.5:4839 , piece_id=5GC664KPRMT7UWO6ZMWLCK66R2BS3VV6YH5TL6UTSHHCYMDKX7RA, seg_pos={0 0}, upload=16602]: Done uploading piece. err: <nil> Jul 19 19:43:08.580746803 (20h52m14.953326158s) [noise=true , address=104.36.82.5:4838 , piece_id=6K6DSZET34IU6GLSQU4OGYI6T754TQKMBUCRWELX6FX7YIZ2DYYQ, seg_pos={0 0}, upload=16602]: Done uploading piece. err: <nil> Jul 19 19:43:08.625021154 (20h52m14.997600509s) [noise=true , address=104.36.82.7:4909 , piece_id=FQ72QZ4Z2RATGSZT4ZF35ZXCDVINOQVYGKGJTCMD6CARBE2WJXKQ, seg_pos={0 0}, upload=16602]: Done uploading piece. err: <nil> Jul 19 19:43:08.631913211 (20h52m15.004492566s) [noise=true , address=45.91.116.9:4261 , piece_id=ORGY5K4DHN3VGKBQKQCWNEF7QC4TSITV2YZUS2Y6LRVRMKXPWAJA, seg_pos={0 0}, upload=16602]: Done uploading piece. err: <nil> Jul 19 19:43:08.646778182 (20h52m15.019357537s) [noise=true , address=104.36.82.16:4295 , piece_id=UMGCMTDS7NLLEVRGPCOZ7OC2W7XG3PAVV4DZT33X2KEEUGLKRURQ, seg_pos={0 0}, upload=16602]: Done uploading piece. err: <nil> Jul 19 19:43:08.705675010 (20h52m15.078254365s) [noise=true , address=45.91.116.13:4154 , piece_id=HU44VLLUBD57S2VJJ6KSBOEQMLAQWC73OXTBUZ2YIWMVSSGZWBAA, seg_pos={0 0}, upload=16602]: Done uploading piece. err: <nil> Jul 19 19:43:08.728943290 (20h52m15.101522635s) [noise=true , address=104.36.82.3:4767 , piece_id=LJOY4D4E626ZKXO5DBNEFWZYJA4SNWB6SQSUYR6E6ZCBCLYDALAA, seg_pos={0 0}, upload=16602]: Done uploading piece. err: <nil> Jul 19 19:43:35.535974912 (20h52m41.908554267s) [noise=true , address=77.90.31.18:4805 , piece_id=BO3EOYNIHA3QR6RWTWDYDE352ORRUXOWXTH4RUN7RP7XMWG7HFCQ, seg_pos={0 0}, upload=16602]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: rpc: dial tcp 77.90.31.18:4805: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:43:35.536028434 (20h52m41.908607789s) [noise=true , address=77.90.31.12:4589 , piece_id=JI6CQS7ZO3BB7A24QIQJAIRQ3O6MHJJSLK5VKGCIVTTIP3FGLGUA, seg_pos={0 0}, upload=16602]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:37498->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:43:35.536147973 (20h52m41.908727328s) [noise=true , address=77.90.31.17:4770 , piece_id=YK3LO2LE5PS25B5U3MJHAAGHJQQR3CBNO2QYBEYZVZ4AXYM6NYQQ, seg_pos={0 0}, upload=16602]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 188.166.225.170:60618->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:43:35.536175526 (20h52m41.908754881s) [noise=true , address=77.90.31.11:4553 , piece_id=BCIRUINTOO4PU3HUXZWNWMUAPF4XJHJGM5JTUKRNSUK6A23Q55BQ, seg_pos={0 0}, upload=16602]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: write tcp 188.166.225.170:57812->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:43:35.536210843 (20h52m41.908790198s) [noise=true , address=77.90.31.10:4517 , piece_id=ECRVBANP3AQAWZBV3FEWF6FXWNHHZNGGOJXARJHFOKQMMPPM6V7Q, seg_pos={0 0}, upload=16602]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: rpc: dial tcp 77.90.31.10:4517: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:43:35.536233497 (20h52m41.908812852s) [seg_pos={0 0}, upload=16602]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 19:43:35.775181158 (20h52m42.147760513s) [upload=16602]: Done uploading segments... Jul 19 19:43:35.775208220 (20h52m42.147787575s) [upload=16602]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.241622513s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.249591306s FAIL (2025-07-19 19:42:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 6s) [upload=16594]: Done starting upload Jul 19 19:42:28.294365609 (20h51m34.666944964s) [upload=16594]: Uploading segments... Jul 19 19:42:28.294589758 (20h51m34.667169113s) [upload=16594]: Got next segment. Inline: false Jul 19 19:42:28.294617531 (20h51m34.667196886s) [upload=16594]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:42:28.294621899 (20h51m34.667201254s) [upload=16594]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:42:28.528525911 (20h51m34.901105266s) [seg_pos={0 0}, upload=16594]: Begin upload segment... Jul 19 19:42:28.528569174 (20h51m34.901148529s) [seg_pos={0 0}, upload=16594]: Done begin upload segment. Jul 19 19:42:28.528590846 (20h51m34.901170201s) [noise=true , address=45.91.116.14:4191 , piece_id=2MPCLLXQM4US3CQTSFT22DXSRHVP4DBIM64RQCAIY5KDJCSH2MSA, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.528636562 (20h51m34.901215917s) [noise=true , address=45.91.116.10:4083 , piece_id=KWNF5U4RHXD63EDZZYGZUSZN525MYM7KTHSLN67GLFDKKZ5FBQVQ, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.528934773 (20h51m34.901514128s) [noise=true , address=77.90.31.12:4589 , piece_id=B6246PN5GEHKO4RMMKRYWKDYV2OMLU5KKFKWV52CPH5QBEJYP7PQ, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.529008214 (20h51m34.901587559s) [noise=true , address=104.36.82.13:4187 , piece_id=LFYKF4QL2FAHP3BBBM2Q374DPHFWZAD7PTX5O5J5KLVJLFP2Q5CQ, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.529672665 (20h51m34.90225202s) [noise=true , address=104.36.82.16:4294 , piece_id=P4XORK44ZOVQYCGLPNMXCJPIZYYYEQO4V3BIIAT76NTZIGQBXP2A, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.529736988 (20h51m34.902316343s) [noise=true , address=104.36.82.15:4258 , piece_id=DXAFAR2AFUSMGAZGTL5UIUJPTIFKMBD774H4OSRYDDQUXFOK43MQ, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.530179165 (20h51m34.90275852s) [noise=true , address=104.36.82.14:4223 , piece_id=Z5LIJPJFYJPYH3HQFND4NUKEMWX4QDYFQ27FGTGPK4SIVOYFKSFQ, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.530387022 (20h51m34.902966377s) [noise=true , address=45.91.116.6:4974 , piece_id=GHNNCWWJDW4TMSJGQEXNY327TD67PULYINVJDWGRMGS26FCWJVTQ, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.530939619 (20h51m34.903518974s) [noise=true , address=77.90.31.13:4628 , piece_id=MKTMGR47ZLVLVAPLKIRW5ZV474OEY25KJOSGHKHDPSLRRIYXRPQQ, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.531167595 (20h51m34.90374695s) [noise=true , address=77.90.31.21:4878 , piece_id=YU2DSBNEMCRBZDZ7TZ7PPWODHOCCESUKRIJMCKU42IJFFX7WHSRQ, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.531591346 (20h51m34.904170691s) [noise=true , address=45.91.116.10:4082 , piece_id=DSMAA7DJGTQ7DY6UMMQ4X7MNGDBXYX36RDGXVLKN65FGTXGZYCZQ, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.532005428 (20h51m34.904584784s) [noise=true , address=77.90.31.19:4841 , piece_id=LUV63FJYCFAYWT36C6UY3RZJM4P3V7Q4U5CC4WPC7332TMOPUR4Q, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.532084400 (20h51m34.904663755s) [noise=true , address=77.90.31.14:4662 , piece_id=UN6ZIUKKGPOGKE6SM6ZGIUD4PUYYJFWOYY4KVURXVNPZ57UPZUDQ, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.532440311 (20h51m34.905019666s) [noise=true , address=104.36.82.16:4293 , piece_id=ETHNJIDFQMVVXAYRAY6QD6RJYDWAIKOI5KVOED7KH4WOVSLSE7GQ, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.532981966 (20h51m34.905561321s) [noise=true , address=45.91.116.5:4298 , piece_id=KZO7I7NDA2NC5CRJUWLS7S2R3ET3RYV2ESPKRAS7SAU2PREQO6CA, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:28.533244449 (20h51m34.905823794s) [noise=true , address=104.36.82.6:4873 , piece_id=S2PIPCLBAGKR3UGX3OGZVBBUGXZDMIETTTWIG6BM6DIUGOBDOQGA, seg_pos={0 0}, upload=16594]: Uploading piece... Jul 19 19:42:30.232190900 (20h51m36.604770255s) [upload=16594]: Next returned nil segment Jul 19 19:42:30.232206991 (20h51m36.604786346s) [upload=16594]: Waiting for error group managing segments... Jul 19 19:42:30.574137701 (20h51m36.946717046s) [noise=true , address=104.36.82.16:4293 , piece_id=ETHNJIDFQMVVXAYRAY6QD6RJYDWAIKOI5KVOED7KH4WOVSLSE7GQ, seg_pos={0 0}, upload=16594]: Done uploading piece. err: <nil> Jul 19 19:42:30.582682190 (20h51m36.955261595s) [noise=true , address=104.36.82.16:4294 , piece_id=P4XORK44ZOVQYCGLPNMXCJPIZYYYEQO4V3BIIAT76NTZIGQBXP2A, seg_pos={0 0}, upload=16594]: Done uploading piece. err: <nil> Jul 19 19:42:30.601280445 (20h51m36.9738598s) [noise=true , address=104.36.82.6:4873 , piece_id=S2PIPCLBAGKR3UGX3OGZVBBUGXZDMIETTTWIG6BM6DIUGOBDOQGA, seg_pos={0 0}, upload=16594]: Done uploading piece. err: <nil> Jul 19 19:42:30.669785776 (20h51m37.042365131s) [noise=true , address=104.36.82.15:4258 , piece_id=DXAFAR2AFUSMGAZGTL5UIUJPTIFKMBD774H4OSRYDDQUXFOK43MQ, seg_pos={0 0}, upload=16594]: Done uploading piece. err: <nil> Jul 19 19:42:30.681614418 (20h51m37.054193773s) [noise=true , address=104.36.82.13:4187 , piece_id=LFYKF4QL2FAHP3BBBM2Q374DPHFWZAD7PTX5O5J5KLVJLFP2Q5CQ, seg_pos={0 0}, upload=16594]: Done uploading piece. err: <nil> Jul 19 19:42:30.686242614 (20h51m37.058821969s) [noise=true , address=45.91.116.6:4974 , piece_id=GHNNCWWJDW4TMSJGQEXNY327TD67PULYINVJDWGRMGS26FCWJVTQ, seg_pos={0 0}, upload=16594]: Done uploading piece. err: <nil> Jul 19 19:42:30.699349802 (20h51m37.071929197s) [noise=true , address=45.91.116.5:4298 , piece_id=KZO7I7NDA2NC5CRJUWLS7S2R3ET3RYV2ESPKRAS7SAU2PREQO6CA, seg_pos={0 0}, upload=16594]: Done uploading piece. err: <nil> Jul 19 19:42:30.719512499 (20h51m37.092091854s) [noise=true , address=45.91.116.10:4083 , piece_id=KWNF5U4RHXD63EDZZYGZUSZN525MYM7KTHSLN67GLFDKKZ5FBQVQ, seg_pos={0 0}, upload=16594]: Done uploading piece. err: <nil> Jul 19 19:42:30.745534798 (20h51m37.118114153s) [noise=true , address=45.91.116.10:4082 , piece_id=DSMAA7DJGTQ7DY6UMMQ4X7MNGDBXYX36RDGXVLKN65FGTXGZYCZQ, seg_pos={0 0}, upload=16594]: Done uploading piece. err: <nil> Jul 19 19:42:30.789634605 (20h51m37.16221395s) [noise=true , address=104.36.82.14:4223 , piece_id=Z5LIJPJFYJPYH3HQFND4NUKEMWX4QDYFQ27FGTGPK4SIVOYFKSFQ, seg_pos={0 0}, upload=16594]: Done uploading piece. err: <nil> Jul 19 19:42:30.797217403 (20h51m37.169796798s) [noise=true , address=45.91.116.14:4191 , piece_id=2MPCLLXQM4US3CQTSFT22DXSRHVP4DBIM64RQCAIY5KDJCSH2MSA, seg_pos={0 0}, upload=16594]: Done uploading piece. err: <nil> Jul 19 19:42:57.648106964 (20h52m4.020686309s) [noise=true , address=77.90.31.14:4662 , piece_id=UN6ZIUKKGPOGKE6SM6ZGIUD4PUYYJFWOYY4KVURXVNPZ57UPZUDQ, seg_pos={0 0}, upload=16594]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: write tcp 188.166.225.170:43362->77.90.31.14:4662: use of closed network connection; write tcp 188.166.225.170:43362->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43362->77.90.31.14:4662: use of closed network connection Jul 19 19:42:57.648130458 (20h52m4.020709813s) [noise=true , address=77.90.31.21:4878 , piece_id=YU2DSBNEMCRBZDZ7TZ7PPWODHOCCESUKRIJMCKU42IJFFX7WHSRQ, seg_pos={0 0}, upload=16594]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: write tcp 188.166.225.170:36154->77.90.31.21:4878: use of closed network connection; write tcp 188.166.225.170:36154->77.90.31.21:4878: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36154->77.90.31.21:4878: use of closed network connection Jul 19 19:42:57.648168531 (20h52m4.020747886s) [noise=true , address=77.90.31.12:4589 , piece_id=B6246PN5GEHKO4RMMKRYWKDYV2OMLU5KKFKWV52CPH5QBEJYP7PQ, seg_pos={0 0}, upload=16594]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: write tcp 188.166.225.170:40146->77.90.31.12:4589: use of closed network connection; write tcp 188.166.225.170:40146->77.90.31.12:4589: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40146->77.90.31.12:4589: use of closed network connection Jul 19 19:42:57.648191205 (20h52m4.02077056s) [noise=true , address=77.90.31.13:4628 , piece_id=MKTMGR47ZLVLVAPLKIRW5ZV474OEY25KJOSGHKHDPSLRRIYXRPQQ, seg_pos={0 0}, upload=16594]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 188.166.225.170:57802->77.90.31.13:4628: use of closed network connection; write tcp 188.166.225.170:57802->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57802->77.90.31.13:4628: use of closed network connection Jul 19 19:42:57.648208237 (20h52m4.020787592s) [noise=true , address=77.90.31.19:4841 , piece_id=LUV63FJYCFAYWT36C6UY3RZJM4P3V7Q4U5CC4WPC7332TMOPUR4Q, seg_pos={0 0}, upload=16594]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: write tcp 188.166.225.170:44606->77.90.31.19:4841: use of closed network connection; write tcp 188.166.225.170:44606->77.90.31.19:4841: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44606->77.90.31.19:4841: use of closed network connection Jul 19 19:42:57.648231301 (20h52m4.020810656s) [seg_pos={0 0}, upload=16594]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jul 19 19:42:57.895222032 (20h52m4.267801397s) [upload=16594]: Done uploading segments... Jul 19 19:42:57.895369914 (20h52m4.267949279s) [upload=16594]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.249618238s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.995950586s FAIL (2025-07-19 17:14:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.995976335s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 6.115593081s FAIL (2025-07-19 17:14:15 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.115623059s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 5.826200384s FAIL (2025-07-19 17:13:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.826293221s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 5.790302755s FAIL (2025-07-19 17:13:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.790338674s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 5.794341923s FAIL (2025-07-19 17:12:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.794370037s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 5.733265822s FAIL (2025-07-19 17:11:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.733293345s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 5.900295296s FAIL (2025-07-19 17:10:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.900320634s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 5.745293034s FAIL (2025-07-19 17:09:33 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.745314514s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 6.029961461s FAIL (2025-07-19 17:08:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.029984304s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 5.918423338s FAIL (2025-07-19 17:07:52 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.918452613s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 5.920597912s FAIL (2025-07-19 17:07:11 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.920630483s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 6.017459663s FAIL (2025-07-19 17:06:29 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.017481415s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)